TL;DR: New llama.cpp release implements CUDA GGML_OP_LIGHTNING_INDEXER with vector and WMMA kernels, improving performance for certain operations.
Summary: The latest llama.cpp release (b10032) introduces a CUDA implementation of the GGML_OP_LIGHTNING_INDEXER operation, featuring both a generic vector kernel and a Turing-capable WMMA kernel. The update includes alignment checks, template parameter optimizations, and cleanup, contributed by Stanisław Szymczyk and the ggml-org team.
Why it matters: For AI builders using llama.cpp on NVIDIA GPUs, this new operator can accelerate specific inference tasks. Consider testing the b10032 release to benchmark any performance gains in your local LLM workflows.
Source: github_releases