TL;DR: llama.cpp released a fix preventing crashes during context shifts for unquantized K cache, improving stability for local LLM inference.
Summary: ggerganov/llama.cpp's b10690 release addresses a memory-related crash during context shifts. The fix ensures the Hadamard matrix is copied to the k_rot tensor only when a buffer is assigned, specifically preventing issues with unquantized K cache.
Why it matters: This update enhances the stability and reliability of running large language models locally using llama.cpp. Developers should update to avoid potential crashes when managing model context, especially with larger or unquantized models.
Source: github_releases