TL;DR: llama.cpp introduced persistent reordered weight layouts for IQ3_S and IQ3_XXS quantization on SYCL, significantly improving inference speed for specific models on Intel Arc GPUs.
Summary: The latest llama.cpp update (b11636) implements persistent reordered weight layouts for IQ3_S and IQ3_XXS quantization schemes on SYCL-compatible hardware. This optimization reorders weights once at load time, allowing for more efficient MMVQ dot products using dp4a and SWAR sign expansion. The change, developed by ggerganov, resulted in up to an 82% increase in token generation speed for models like Qwen3.8-27B-GSQ-RCO-IQ3_S on Intel Arc Pro B70 GPUs.
Why it matters: This update offers substantial performance gains for AI builders deploying quantized models on Intel Arc GPUs via llama.cpp, making local inference faster and more efficient. Developers should consider using IQ3_S/IQ3_XXS quantization for their models to leverage these new SYCL optimizations.
Source: github_releases