Best Local LLM 2026: Models, RAM Needs and Runtimes

The Bottom Line#

The size of your RAM decides which local LLM you can run, and a single rule gets you most of the way: at 4-bit quantization a model needs roughly 0.5 to 0.6 GB of memory per billion parameters. A 7B model lands around 4 to 5 GB, a 14B model around 9 to 10 GB, a 32B model around 20 GB.

That rule covers the model weights only. The KV cache, activations and runtime overhead come on top, and they grow with your context length. A model that fits on paper can still run out of memory in practice, which is why the classes below leave headroom instead of filling the machine to the last gigabyte.

The second decision matters just as much and gets far less attention: which runtime you use to serve the model. That is where this guide differs from most. We track 12 open-source local-inference projects with a license check on each, and the picture there changed a lot in 2026.

How Much RAM Do You Need to Run an LLM Locally?#

Take your available memory, subtract what the operating system and your other applications need, then apply the 0.5 to 0.6 GB per billion parameters rule to what is left.

A note on the widespread assumption that a 4-bit model always fits in the next smaller GPU: that is wrong. Memory is not only weights. The sources listed at the end are explicit that KV cache, activations and runtime overhead can push real consumption well past the arithmetic, especially at long context lengths.

Which Open Models Are Worth Running Locally?#

The open-weight landscape moves fast enough that any specific ranking ages within months. What holds up across the size classes as of August 2026:

• Around 7-8B is the entry point that runs on almost any modern laptop. Apache-2.0 licensed models in this class are plentiful, which matters if you intend to use the output commercially.
• Around 14B is where local models start to feel genuinely useful rather than merely impressive, and it is the sweet spot for 16 GB machines.
• Around 30-32B is where the gap to hosted frontier models narrows enough that many workflows stop needing the cloud at all.

Check the license before you build on a model. "Open weights" and "open source" are not the same thing, and several widely used models ship under bespoke licenses with usage restrictions rather than Apache-2.0 or MIT.

Which Runtime Should You Use?#

This is the part where our own data goes further than a general roundup. Every project below is tracked in the Open-Source AI Radar with a manual license check, and the star counts carry the date they were taken.

Star counts taken between 2026-06-05 and 2026-06-26; the exact date per project is on its detail page. All ten are OSI-approved licenses, which is not something you can assume in this space.

If you do not know where to start, run whichllm first. It reads your hardware and tells you which models will run well, which removes the guesswork the RAM table above can only approximate.

Which Local LLM Is Best for Coding?#

Coding is the workload where the size class matters most, because a model that is merely adequate at chat becomes actively unhelpful when it invents an API that does not exist. The practical floor is higher than for general use.

• Below 14B, treat a local model as autocomplete rather than a collaborator. It will finish the line you started and it will save keystrokes, but it will not reason about your codebase.
• At 14B, single-file work becomes realistic: writing a function to a description, explaining unfamiliar code, translating between languages.
• At 30-32B, multi-file reasoning and agent workflows start to hold together. This is where running locally stops being a compromise for most day-to-day work.

Context length matters as much as parameter count here, and it is the part people underestimate. Feeding a model three files instead of one multiplies the KV cache, and that memory comes out of the same budget as the weights. Plan for a class smaller than your RAM table suggests if you intend to work with long context.

On the tooling side, two of the tracked projects are built specifically for this. Rapid-MLX is an OpenAI-compatible MLX server designed for coding agents rather than chat, and claude-code-local proxies the Claude Code CLI to a local model so the workflow stays identical while nothing leaves the machine.

Is Self-Hosting an LLM Worth It?#

It depends on which of three reasons applies to you, and only one of them is about money.

Privacy is the strongest case. If your prompts contain client data, patient records, unreleased code or anything under an NDA, a local model removes the question entirely rather than answering it with a vendor's data processing agreement. No configuration to get wrong, no policy change to track.

Cost only works at volume. A single seat on a hosted frontier model runs around 20 USD a month. To beat that with hardware you need either machines you already own or usage high enough that per-token pricing hurts. Buying a GPU to save a subscription is not a saving.

Offline operation is niche but absolute. Air-gapped environments, field work, unreliable connectivity: either the model runs locally or there is no model.

What you give up is capability at the top end and the maintenance being someone else's problem. You are now responsible for updates, quantization choices and the day a new model release makes your setup look dated. For most people the honest answer is a hybrid: local for the routine and the sensitive, hosted for the hard problems.

Ollama Alternatives Worth Knowing#

Ollama is the default answer and a reasonable one. The projects above matter when its defaults stop fitting:

• You are on Apple Silicon and want the hardware fully used. oMLX, Rapid-MLX and vllm-mlx build on MLX rather than a cross-platform abstraction.
• You want one binary with no runtime to manage. shimmy is a single Rust executable with an OpenAI-compatible API.
• You are serving agents, not chatting. tokenspeed targets throughput for agentic workloads rather than single-user latency.
• You want more than an LLM. DreamServer installs voice, RAG and image generation alongside the model in one step.
• You want Claude Code without the cloud. claude-code-local proxies the CLI to a local model.

What This Guide Does Not Cover#

Being explicit about the limits, because a lot of comparable pages are not:

• No benchmark numbers of our own. We have not run tokens-per-second measurements on our own hardware. The RAM figures are the published quantization rule of thumb, not our measurements.
• No specific model ranking. Open-weight leaderboards shift monthly, and a ranking published today would mislead within a quarter. The size classes hold up; the individual names do not.
• Apple Silicon is over-represented in the runtime table. That reflects where open-source local inference actually concentrated in 2026, not a preference on our part. If you are on an NVIDIA GPU, tokenspeed and shimmy are the relevant entries.
• Nothing here is affiliate-linked. These are open-source projects. We earn nothing from them, which is also why the list is not padded.

FAQ#

What is the best local LLM for 8 GB of RAM?#

A model in the 7 to 8 billion parameter class, quantized to 4-bit. That comes to roughly 4 to 5 GB of weights and leaves room for the KV cache and your operating system. Below that class, quality drops off sharply for anything beyond simple summarising.

How much VRAM do I need for a 32B model?#

Around 19 to 21 GB for the weights at 4-bit quantization, which in practice means a 32 GB machine rather than a 24 GB one. The gap between the arithmetic and the real requirement is the KV cache and activations, and it widens as your context length grows.

Is a local LLM as good as ChatGPT or Claude?#

Not at the top end. A 32B model running locally closes much of the gap for summarising, drafting and routine code work, but hosted frontier models remain ahead on hard reasoning and long-context tasks. The reason to run locally is usually privacy, cost at volume, or offline operation, not raw capability.

What is the difference between open source and open weights?#

Open weights means you can download and run the model. Open source, in the sense the OSI uses, additionally means the license permits use, modification and redistribution without bespoke restrictions. Several prominent models are open weights under a custom license that limits commercial use or scale. Check the license text, not the marketing.

Do I need a GPU to run an LLM locally?#

No, but it changes what is realistic. Apple Silicon machines run models on unified memory without a discrete GPU and do so well, which is why so much of the 2026 tooling targets them. On a CPU-only x86 machine a 7B model will run, slowly enough that it suits batch work rather than conversation.

Sources#

• Spheron: GPU memory requirements for LLMs
• Plugable: VRAM requirements for local LLMs
• n4n.ai: How VRAM requirements scale with parameter count
• OS Foundry: LLM quantization explained (GGUF)
• Hugging Face: State of open models, summer 2026

Runtime data from our own Open-Source AI Radar, where each project carries its license check and the date its star count was taken.