TL;DR: A new open benchmark ranks model, inference engine, and agent-harness combinations for running coding agents entirely on local hardware.
Summary: GitHub repo evanwtf/local-llm publishes real-repo coding-agent benchmarks that score combinations of model, inference engine, and agent harness by pass rate and wall time. Testing runs on an M5 Max MacBook Pro and a two-node DGX Spark cluster, with OpenCode as the primary harness and engines including llama.cpp, MLX, Ollama, and vLLM. The goal is to identify which full stack actually performs best for agentic coding without cloud inference.
Why it matters: Local-first agent stacks are becoming viable, and this gives builders a concrete comparison of engine and harness choices rather than raw model benchmarks. Watch the pass-rate and wall-time tables before committing to an inference stack for on-device coding agents.
Source: github_topics