TL;DR: llama.cpp b9990 introduces speculative decoding support for Minimax2 EAGLE3, enabling faster inference with custom draft models.
Summary: The latest llama.cpp release (b9990) adds spec decoding support for the Minimax2 EAGLE3 draft model, including a fix for a nullptr bug in the EAGLE3 implementation. This allows users to leverage the EAGLE3 speculative decoding method with Minimax2 models for improved token generation speed.
Why it matters: For AI builders running local LLMs, speculative decoding can significantly reduce latency. Now those using Minimax2 models can experiment with EAGLE3 drafts. Try updating to b9990 and configure the --speculative-model flag with an EAGLE3-compatible draft model.
Source: github_releases