Your hardware in, the right local model out. Pick your GPU or Mac and get the best LLMs that actually fit, the right quantization, and a copy-paste Ollama command.
Estimates, not benchmarks. Model sizes are the download sizes listed on ollama.com. Memory needed adds headroom for about 8k tokens of context. Speeds are rough figures based on memory bandwidth. Real results vary with context length, drivers, background apps and Ollama version. Data updated .
A step-by-step guide for your exact machine covering install, the best models, context settings and speed tuning.
One short email when a new open model fits your hardware. No spam.
We take the model's real download size from ollama.com and add headroom for context and runtime. We then compare that to your usable fast memory: VRAM on a GPU, or about 67–75% of unified memory on a Mac, because macOS limits how much the GPU can use. We pick the highest-quality quantization that fits.
They're compression levels. Q4_K_M is the sweet spot for most people, with about 4.8 bits per weight and small quality loss. Q8_0 is close to full quality but roughly twice the size. When a bigger model fits at Q4, it usually beats a smaller model at Q8.
On PC GPUs, Ollama can spill layers into system RAM. It works, but it's much slower, so we flag those results and rank them lower.
Ollama is the easiest. LM Studio is a friendly desktop app (MLX on Mac, GGUF elsewhere). llama.cpp gives you the most control and runs the same GGUF files. MLX is Apple's own framework and is often the fastest on Apple Silicon. vLLM is a high-throughput server for NVIDIA/AMD: it reserves about 90% of VRAM up front and can't offload to system RAM, so fewer models fit. We hide runtimes that don't run on your hardware. The picker is saved in the link (?rt=).
Hugging Face. GGUF files come from unsloth, bartowski or ggml-org, MLX builds from mlx-community, and vLLM uses the official or AWQ/GPTQ/FP8 repos. Every repo and file size is checked against the Hugging Face API, and anything we couldn't verify is left out. Sizes vary a little between runtimes. For example, MLX 4-bit builds are usually a bit smaller than Q4_K_M GGUFs.
Install Ollama from ollama.com, open a terminal and paste the command from your top result. The first run downloads the model.
The model list and Ollama tags are re-checked against ollama.com regularly. Last update: .