TL;DR: llama.cpp updated its SYCL backend to increase the minimum buffer size for USM system allocations, improving performance for large models on devices with limited VRAM.
Summary: The latest llama.cpp release (b10017) raises the minimum buffer size for SYCL USM system allocations from 1 GiB to 4 GiB. This change, based on experiments with models like Qwen3.5-35B-A3B-Q8 on Intel B70, aims to optimize memory overcommitment when model weights exceed available VRAM.
Why it matters: This enhancement improves memory management for running large language models on consumer hardware, particularly for Intel SYCL users. Developers should observe better stability and performance when deploying memory-intensive models on devices with constrained VRAM.
Source: github_releases