TL;DR: llama.cpp now supports IM2COL_3D on Hexagon DSPs, improving performance for convolutional operations on Qualcomm hardware.
Summary: The latest llama.cpp update (b11560) introduces IM2COL_3D support for Hexagon DSPs. This enhancement, co-authored by Max Krasnyansky from Qualcomm, includes optimized DMA paths, rewritten kernel logic, and improved handling of non-contiguous views for efficient convolutional operations.
Why it matters: This significantly boosts performance for running LLMs on devices powered by Qualcomm Hexagon DSPs, enabling faster on-device inference. Developers should explore leveraging llama.cpp for edge AI applications on Qualcomm platforms.
Source: github_releases