whichllm

Picking a local LLM is harder than it should be. Most advice collapses into parameter count: people assume a 70B model beats a 7B one, then discover it will not fit in their VRAM or runs at a few tokens per second. whichllm, a CLI by Andyyyy64, tackles that gap. It detects your hardware (GPU, CPU, RAM) and ranks the models that will actually run well on your specific machine, scored against real leaderboards rather than size alone. The pitch is simple: instead of guessing whether a quantized 32B model is a better bet than a smaller, newer one, you get an evidence-based ranking tied to your actual constraints. It is a decision tool for the messy middle ground between raw benchmarks and what your box can realistically handle.

Hardware detection runs across NVIDIA (via nvidia-ml-py), AMD on Linux, Apple Silicon via Metal, and CPU-only setups with AVX checks. The ranking is where the project gets interesting. It aggregates a live tier (LiveBench, Artificial Analysis, Aider) and a frozen tier (Open LLM Leaderboard v2, Arena ELO, vision indexes), then scores models on benchmark quality, size, quantization penalties, evidence confidence, runtime fit, and estimated speed. Notably, it is honest about uncertainty: markers flag estimated, self-reported, or missing data, and stale leaderboards get dampened along model lineages so old scores do not inflate newer variants. Beyond plain ranking, commands like plan ask which GPU a target model needs, upgrade compares two cards, and run downloads and chats with a model immediately.

The natural audience is anyone running models locally who is tired of trial and error: developers, researchers, and hardware enthusiasts who want a defensible answer rather than a forum opinion. Concrete use cases stand out. Before buying a card, whichllm --gpu "RTX 5090" simulates that hardware and shows what it would unlock, and plan "llama 3 70b" works backward from a model to its requirements. For automation, --json output piped through jq lets you select the top model_id programmatically, useful in provisioning scripts. Task profiles such as --profile coding or --profile vision narrow results to what you care about, and snippet prints copy-paste Python to get going. It rewards people who treat model selection as a real decision with budget and latency tradeoffs.

The caveats matter, and the project is refreshingly upfront about most of them. Speed figures are estimates with confidence ranges, not measured guarantees: actual throughput depends on your backend, quantization, and context length, so treat tok/s numbers as guidance rather than promises. The Ollama path is rough, since Ollama model names do not reliably match HuggingFace repo IDs, leaving a manual mapping step before you can ollama run a recommendation. It is Python 3.11+ only, and the live benchmark tiers depend on external sources staying available and current, with caching that can lag. Most importantly, it is early, at a 0.x version, so scoring weights and data coverage will keep shifting. None of this is fatal, but it is a planning aid, not a benchmark harness.

Versus a commercial alternative there is not really a direct paid equivalent: the comparison is against manually trawling leaderboards, Reddit threads, and VRAM calculators yourself, which is exactly the friction whichllm compresses. That makes it genuinely useful at the moment of decision, when you are choosing what to run or which GPU to buy and want a ranked starting point grounded in multiple benchmarks rather than a single number. Adopt it as a fast first pass and a sanity check on your shortlist, then validate the top one or two picks on your own machine. Where it is too early is if you need measured, reproducible throughput before committing: for that, install the model and benchmark it directly. As a triage tool, though, it earns its place.