The gist

whichllm is an open-source command-line tool created by Andyyyy64. It solves the problem of selecting an optimal large language model (LLM) for local use by automatically detecting a user's hardware (GPU, CPU, RAM). The tool then queries HuggingFace and ranks models based on a combination of performance benchmarks, VRAM fit, and recency, ensuring users get evidence-based recommendations instead of just the largest model that fits their machine.

What it does

Auto-detects NVIDIA, AMD, or Apple Silicon hardware to assess system capabilities.
Ranks local LLMs from HuggingFace using a weighted score from real-world benchmarks.
Simulates hardware performance for any GPU to help plan future purchases.
Determines the necessary hardware requirements for a specific model like 'llama 3 70b'.
Initiates a chat session with a chosen model using a single 'run' command.
Generates copy-paste Python code snippets to integrate models into projects.

How it works

whichllm is a Python CLI tool installed via pip, brew, or uv. A user runs commands like whichllm or whichllm --gpu "RTX 4090" in their terminal. The tool fetches live model data from the HuggingFace API, analyzes it against the user's hardware profile, and outputs a ranked list of models in a terminal table or as JSON. It considers VRAM, speed, and benchmark scores to make its recommendation. The project is free and open-source.

Best for

This tool is ideal for developers and AI researchers who want to run LLMs locally and need to identify the highest-performing model that will run efficiently on their specific hardware configuration.

Watch out for

The reported speed (tokens/second) is an estimation for planning purposes and not a live benchmark of your actual hardware performance.