TL;DR: llama.cpp has updated its HIP backend to optimize matrix multiplication for specific quantized model types on RDNA3.5 GPUs, leading to significant performance gains.
Summary: ggerganov/llama.cpp release b11618 introduces an optimization for AMD RDNA3.5 GPUs (gfx115x) by adjusting the matrix multiplication (MMQ) tile configuration. Specifically, it extends the use of 64-row MMQ tiles for certain quantized model types, allowing for better utilization of on-chip memory (LDS) and enabling a second workgroup per compute unit.
Why it matters: This update means faster inference for quantized large language models on AMD RDNA3.5 GPUs, crucial for indie developers and AIGC creators using consumer hardware. Builders should update llama.cpp to leverage these performance improvements for their local AI applications.
Source: github_releases