llama.cpp b9966 speeds up tensor splitting with static regex

Coding OpenSource

TL;DR: llama.cpp b9966 fixes a performance bottleneck where regex patterns were recompiled on every tensor split call, now static to reduce overhead.

Summary: llama.cpp release b9966 includes a fix for tensor-split regex patterns. Previously, the function llama_meta_device_get_split_state() recompiled 29 std::regex objects on every call, dominating decode thread time. The fix makes them static const, compiled once, with behavior unchanged.

Why it matters: This optimization improves inference throughput for multi-GPU setups using tensor splitting. Update to b9966 if you use llama.cpp with -sm tensor split mode.

Source: github_releases