TL;DR: Llama.cpp has added architectural support for cohere2-MoE, enabling developers to run Cohere's new North Mini Code 30B MoE model locally with high efficiency.
Summary: A new pull request in llama.cpp introduces architectural support for cohere2-MoE, enabling local execution of Cohere's North-Mini-Code-1.0 model. Developed by Cohere Labs, North Mini Code is an open-weights 30B total (3B active) parameter mixture-of-experts model optimized for code generation and agentic software engineering. The model supports a 256K context length and a 64K max output limit.
Why it matters: This enables builders to run powerful, agent-optimized coding models locally on standard hardware without relying on proprietary APIs. Developers should recompile llama.cpp and try out the quantized GGUF weights.
Source: r/localllama