← CanIRunLocal · All hardware · All models
With 8 GB VRAM, assuming 16 GB system RAM for partial offload, these are the best open models that fit. The top pick is Bonsai 2 9B (ternary) at Q2_0 (~4.6 GB, ~45 tok/s): ollama run bonsai-2:9b.
Best for coding: LFM2.5 8B-A1B (MoE) (lfm2.5:8b), Granite 4.2 8B (granite4.2:8b-q5_K_M), Qwen3.5 4B (qwen3.5:4b-q8_0).
Runtimes available for this hardware: Ollama, LM Studio, llama.cpp, vLLM.
Open the interactive checker for RTX 3060 8GB| # | Model | Quant | Memory needed | Fit | Speed | Command |
|---|---|---|---|---|---|---|
| 1 | Bonsai 2 9B (ternary) | Q2_0 | 4.6 GB | fits in fast memory | ~45 tok/s | ollama run bonsai-2:9b |
| 2 | LFM2.5 8B-A1B (MoE) | Q4_K_M | 6 GB | fits in fast memory | ~117 tok/s | ollama run lfm2.5:8b |
| 3 | Granite 4.2 8B | Q5_K_M | 7.3 GB | fits in fast memory | ~23 tok/s | ollama run granite4.2:8b-q5_K_M |
| 4 | Nimble v3 9B (Bespoke Labs, decision) | Q4_K_M | 6.6 GB | fits in fast memory | ~26 tok/s | ollama run nimble-v3:9b-q4_K_M |
| 5 | Qwen3 8B | Q4_K_M | 6.2 GB | fits in fast memory | ~28 tok/s | ollama run qwen3:8b |
| 6 | DeepSeek R1 0528 Qwen3 8B | Q4_K_M | 6.2 GB | fits in fast memory | ~28 tok/s | ollama run deepseek-r1:8b |
| 7 | DeepSeek R1 Distill Qwen 7B | Q4_K_M | 5.7 GB | fits in fast memory | ~31 tok/s | ollama run deepseek-r1:7b |
| 8 | Ministral 3 8B | Q4_K_M | 7.4 GB | fits in fast memory | ~24 tok/s | ollama run ministral-3:8b |
| 9 | Qwen3-VL 8B | Q4_K_M | 7.5 GB | fits in fast memory | ~24 tok/s | ollama run qwen3-vl:8b |
| 10 | MiniCPM-V 2.6 8B | Q4_K_M | 7.1 GB | fits in fast memory | ~25 tok/s | ollama run minicpm-v:8b-2.6-q4_K_M |
| 11 | Qwen 2.5 VL 7B | Q4_K_M | 7.4 GB | fits in fast memory | ~24 tok/s | ollama run qwen2.5vl:7b |
| 12 | Bonsai 2 4B (ternary) | Q2_0 | 2.8 GB | fits in fast memory | ~90 tok/s | ollama run bonsai-2:4b |
| 13 | Qwen3.5 4B | Q8_0 | 6.4 GB | fits in fast memory | ~28 tok/s | ollama run qwen3.5:4b-q8_0 |
| 14 | Gemma 4 E2B (effective 2.3B) | Q8_0 | 7.2 GB | fits in fast memory | ~24 tok/s | ollama run gemma4:e2b-it-q8_0 |
| 15 | Phi-4 Mini 3.8B | Q8_0 | 4.9 GB | fits in fast memory | ~35 tok/s | ollama run phi4-mini:3.8b-q8_0 |
| 16 | Qwen3 4B | Q8_0 | 5.2 GB | fits in fast memory | ~33 tok/s | ollama run qwen3:4b-q8_0 |
| 17 | Granite 4.2 3B | Q8_0 | 4.7 GB | fits in fast memory | ~37 tok/s | ollama run granite4.2:3b-q8_0 |
| 18 | Qwen3-VL 4B | Q8_0 | 6.3 GB | fits in fast memory | ~28 tok/s | ollama run qwen3-vl:4b-instruct-q8_0 |
| 19 | Llama 3.2 3B | Q8_0 | 4.2 GB | fits in fast memory | ~42 tok/s | ollama run llama3.2:3b-instruct-q8_0 |
| 20 | Gemma 3 4B | Q8_0 | 6.2 GB | fits in fast memory | ~29 tok/s | ollama run gemma3:4b-it-q8_0 |
| 21 | tev1 4B (Together AI, decision) | Q8_0 | 5.4 GB | fits in fast memory | ~31 tok/s | ollama run tev1:4b-q8_0 |
| 22 | Ministral 3 3B | Q8_0 | 5.6 GB | fits in fast memory | ~32 tok/s | ollama run ministral-3:3b-instruct-2512-q8_0 |
| 23 | Qwen3.5 2B | Q8_0 | 3.8 GB | fits in fast memory | ~53 tok/s | ollama run qwen3.5:2b-q8_0 |
| 24 | Qwen3 1.7B | Q8_0 | 2.9 GB | fits in fast memory | ~65 tok/s | ollama run qwen3:1.7b-q8_0 |
| 25 | Qwen3-VL 2B | Q8_0 | 3.8 GB | fits in fast memory | ~53 tok/s | ollama run qwen3-vl:2b-instruct-q8_0 |
| 26 | Bonsai 2 27B (ternary) | Q2_0 | 10.6 GB | partial offload (slower) | ~10 tok/s | ollama run bonsai-2:27b |
| 27 | DeepSeek R1 Distill Qwen 1.5B | Q8_0 | 2.6 GB | fits in fast memory | ~76 tok/s | ollama run deepseek-r1:1.5b-qwen-distill-q8_0 |
| 28 | SmolLM2 1.7B | Q8_0 | 2.5 GB | fits in fast memory | ~80 tok/s | ollama run smollm2:1.7b |
| 29 | LFM2.5 24B-A2B (MoE) | Q4_K_M | 16.2 GB | partial offload (slower) | ~24 tok/s | ollama run lfm2.5:24b |
| 30 | Bonsai 2 2B (ternary) | Q2_0 | 2 GB | fits in fast memory | ~158 tok/s | ollama run bonsai-2:2b |
| 31 | gpt-oss 20B (MoE, MXFP4) | MXFP4 | 15.1 GB | partial offload (slower) | ~13 tok/s | ollama run gpt-oss:20b |
| 32 | GLM-OCR 0.9B | BF16 | 3.7 GB | fits in fast memory | ~53 tok/s | ollama run glm-ocr:latest |
| 33 | MiniCPM-V 4.6 1B | Q8_0 | 2.9 GB | fits in fast memory | ~76 tok/s | ollama run minicpm-v4.6:q8_0 |
| 34 | Llama 3.2 1B | Q8_0 | 2 GB | fits in fast memory | ~111 tok/s | ollama run llama3.2:1b |
| 35 | LFM2.5 1.2B Thinking | Q8_0 | 1.9 GB | fits in fast memory | ~120 tok/s | ollama run lfm2.5-thinking:1.2b-q8_0 |
| 36 | Gemma 3 1B | Q8_0 | 1.8 GB | fits in fast memory | ~131 tok/s | ollama run gemma3:1b-it-q8_0 |
| 37 | Llama 3.1 8B | Q5_K_M | 6.7 GB | fits in fast memory | ~25 tok/s | ollama run llama3.1:8b-instruct-q5_K_M |
| 38 | Qwen 2.5 7B | Q5_K_M | 6.4 GB | fits in fast memory | ~27 tok/s | ollama run qwen2.5:7b-instruct-q5_K_M |
| 39 | Qwen 2.5 Coder 7B | Q5_K_M | 6.4 GB | fits in fast memory | ~27 tok/s | ollama run qwen2.5-coder:7b-instruct-q5_K_M |
| 40 | Qwen3.5 0.8B | Q8_0 | 2 GB | fits in fast memory | ~144 tok/s | ollama run qwen3.5:0.8b-q8_0 |
| 41 | Mistral 7B v0.3 | Q5_K_M | 6.1 GB | fits in fast memory | ~28 tok/s | ollama run mistral:7b-instruct-q5_K_M |
| 42 | Code Llama 7B | Q5_K_M | 5.7 GB | fits in fast memory | ~30 tok/s | ollama run codellama:7b-instruct-q5_K_M |
| 43 | LLaVA 1.6 7B | Q5_K_M | 6.7 GB | fits in fast memory | ~27 tok/s | ollama run llava:7b-v1.6-vicuna-q5_K_M |
| 44 | Qwen3 0.6B | Q8_0 | 1.5 GB | fits in fast memory | ~173 tok/s | ollama run qwen3:0.6b-q8_0 |
| 45 | Mistral Nemo 12B | Q4_K_M | 8.7 GB | partial offload (slower) | ~14 tok/s | ollama run mistral-nemo:12b-instruct-2407-q4_K_M |
| 46 | Qwen3.5 9B | Q4_K_M | 8 GB | partial offload (slower) | ~19 tok/s | ollama run qwen3.5:9b-q4_K_M |
| 47 | SmolLM2 360M | FP16 | 1.3 GB | fits in fast memory | ~197 tok/s | ollama run smollm2:360m |
| 48 | Ornith-1.5 9B | Default | 8 GB | partial offload (slower) | ~19 tok/s | ollama run ornith-1.5:9b |
| 49 | DeepSeek R1 Distill Qwen 14B | Q4_K_M | 10.3 GB | partial offload (slower) | ~9 tok/s | ollama run deepseek-r1:14b |
| 50 | Gemma 4 12B | Q4_K_M | 9.6 GB | partial offload (slower) | ~11 tok/s | ollama run gemma4:12b-it-q4_K_M |
| 51 | Gemma 3 12B | Q4_K_M | 9.8 GB | partial offload (slower) | ~11 tok/s | ollama run gemma3:12b |
| 52 | Phi-4 14B | Q4_K_M | 10.4 GB | partial offload (slower) | ~9 tok/s | ollama run phi4:14b |
| 53 | Gemma 4 E4B (effective 4.5B) | Q4_K_M | 7.9 GB | partial offload (slower) | ~19 tok/s | ollama run gemma4:e4b-it-q4_K_M |
| 54 | Llama 3.2 Vision 11B | Q4_K_M | 9.3 GB | partial offload (slower) | ~12 tok/s | ollama run llama3.2-vision:11b |
| 55 | Qwen 2.5 3B | Q8_0 | 4.1 GB | fits in fast memory | ~44 tok/s | ollama run qwen2.5:3b-instruct-q8_0 |
| 56 | Qwen3 14B | Q4_K_M | 10.6 GB | partial offload (slower) | ~9 tok/s | ollama run qwen3:14b |
| 57 | SmolLM2 135M | FP16 | 0.9 GB | fits in fast memory | ~533 tok/s | ollama run smollm2:135m |
| 58 | Ministral 3 14B | Q4_K_M | 10.7 GB | partial offload (slower) | ~9 tok/s | ollama run ministral-3:14b |
| 59 | Gemma 3n E2B (on-device) | Q8_0 | 7.8 GB | partial offload (slower) | ~19 tok/s | ollama run gemma3n:e2b-it-q8_0 |
| 60 | Qwen 2.5 Coder 1.5B | Q8_0 | 2.3 GB | fits in fast memory | ~90 tok/s | ollama run qwen2.5-coder:1.5b-instruct-q8_0 |
| 61 | DeepSeek Coder V2 Lite 16B (MoE) | Q4_K_M | 11 GB | partial offload (slower) | ~26 tok/s | ollama run deepseek-coder-v2:16b-lite-instruct-q4_K_M |
| 62 | Gemma 3n E4B (on-device) | Q8_0 | 10.5 GB | partial offload (slower) | ~9 tok/s | ollama run gemma3n:e4b-it-q8_0 |
| 63 | Mistral Small 24B | Q4_K_M | 15.8 GB | partial offload (slower) | ~4 tok/s | ollama run mistral-small:24b |
| 64 | Devstral Small 2 24B | Q4_K_M | 16.8 GB | partial offload (slower) | ~4 tok/s | ollama run devstral-small-2:24b |
| 65 | Clef-Flash 9B (Cloudflare, decision) | Q8_0 | 12.4 GB | partial offload (slower) | ~6 tok/s | ollama run clef-flash:9b-q8_0 |
| 66 | Qwen 2.5 14B | Q4_K_M | 10.3 GB | partial offload (slower) | ~9 tok/s | ollama run qwen2.5:14b |
| 67 | Qwen 2.5 Coder 14B | Q4_K_M | 10.3 GB | partial offload (slower) | ~9 tok/s | ollama run qwen2.5-coder:14b |
Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-18.
A project by OnDevice Labs · llms.txt