TL;DR: llama.cpp release b11723 enhances Hexagon DSP performance for longer context and SWA models, improving efficiency for on-device AI.
Summary: The latest llama.cpp release, b11723, introduces significant improvements to FLASH_ATTN_EXT on Hexagon DSPs. These updates include support for masked block skipping to reduce DMA fetching and finetuning of hmx->hvx crossover thresholds, specifically boosting Gemma-4 performance.
Why it matters: AI builders targeting edge devices with Hexagon DSPs can expect better performance and efficiency for running large language models. Experiment with this update to see how it impacts your on-device LLM applications, especially those with longer contexts or using SWA models.
Source: github_releases