TL;DR: Unsloth's GGUF of DeepSeek-V4-Flash-0731 runs losslessly on a single 40GB A100 at ~17.7 tok/s with 6 experts in VRAM, enabling full agentic coding.
Summary: A test on a single 40GB A100 shows DeepSeek-V4-Flash-0731 (via unsloth's Q8_K_XL GGUF) generating at 17.7 tok/s with 6 experts loaded into VRAM, using only 15.8GB of the 40GB capacity while all experts run on CPU. The model was driven by Codex through a complete agentic coding loop on a Linux box. This demonstrates lossless inference of a large MoE model on one GPU.
Why it matters: AI builders can now run a high-capability MoE model locally without multi-GPU setups; try unsloth's GGUF to reproduce agentic coding on a single A100.
Source: reddit