TL;DR: llama.cpp build b11700 lands conversion and runtime support for IBM's Granite Switch models, which serve as decision/guardrail models that can also chat.
Summary: PR #30676 by IBM's Gabe Goodhart adds Granite Switch support to llama.cpp, including a new decision.chat conversion template, a Granite Switch decision model enum, common-layer handling for models that both chat and classify, and server plumbing for decision mode. The work follows IBM's granite-switch-4.1-8b usage pattern of running a guardian model and a decision model side by side. This lets local inference of a purpose-built routing/decision model rather than relying on a general chat model for gating.
Why it matters: Builders running local agents can now pair a small dedicated decision model with a chat model for tool routing, moderation, or guardrails without a cloud call. Try the new build with granite-switch-4.1-8b to see whether a local classifier beats prompt-based routing in your pipeline.
Source: github_releases