TL;DR: Taco is a cross-platform LLM compressor that uses integer-only inference to keep compressed model outputs bit-identical across GPUs and CPUs.
Summary: Taco is an LLM compression tool that replaces float operations with integer-only inference—int8 weights, fixed rounding, and lookup tables—so the decoder reproduces the encoder's probabilities exactly. The project includes a float32-to-integer converter and conformance vectors, with proof being a single hash matching across seven backends including Mac CPU/Metal, Linux CPU/CUDA, and Windows CPU/CUDA/Vulkan. The motivation: a float GPT-2 compressed on a Mac decoded zero of four windows when run on x86.
Why it matters: Deterministic compressed inference simplifies deploying the same model across heterogeneous hardware and makes reproducibility auditable. Watch for Taco's integer-only approach to become a standard for model compression pipelines, and try its conformance vectors when validating cross-platform deployment.
Source: reddit