TL;DR: A repo of vLLM patches and a requant pipeline claims Qwen3.8-27B at 150k–262k context on a single 24 GB card, with throughput numbers for anyone serving large models on consumer GPUs.
The Catalyst
syv-ai/HyperQwen ships vLLM patches plus a requantization pipeline for Qwen3.8-27B on one 24 GB card (RTX 3090 class). Reported: 127 tok/s single-user, 381 tok/s when the answer quotes the prompt, ~1,035 tok/s at 64 concurrent requests, at 150k–262k context.
The Blast Radius
Turns a 27B model at long context into something that fits on hardware already in the rack, instead of an A100/H100 rental. Check which vLLM version the patches target and how much KV cache is left at 262k before replacing a working serving config.
Source: github_topics