← CanIRunLocal · All hardware · All models

Can I run d3 4B (vLLM Semantic Router, decision) locally?

vLLM Semantic Router · 4B parameters · chat, vision.

Ollama weight files: Q8_0 5.8 GB (d3:4b-q8_0).

d3 scores a choice, a yes/no, or a rating from text, structured state, or a still image. It does not write a chat reply, and it needs Ollama 0.40.9 or later. Apache-2.0. d3:4b and d3:latest are the same alias. On Apple Silicon that alias is the MXFP8 package; on other systems it is Q8_0. The fit list uses the Q8_0 files: d3:2b-q8_0 3.2GB, d3:4b-q8_0 5.8GB, d3:9b-q8_0 10GB, d3:27b-q8_0 30GB. MLX tags (mxfp8, nvfp4) are left out, same as other models. The tag list shows a 256K window; these packages run a 4,096-token context.

Yes on 77 machines in this list. The smallest is RTX 3060 8GB: Q8_0, about 6.9 GB, roughly 25 tokens per second, via Ollama. ollama run d3:4b-q8_0

• DGX Spark 128GB: Q8_0, ~6.9 GB, fits in fast memory, ~35 tok/s — ollama run d3:4b-q8_0
• Mac Studio M5 Max 128GB: Q8_0, ~6.9 GB, fits in fast memory, ~85 tok/s — ollama run d3:4b-q8_0
• Mac Studio M5 Ultra 256GB: Q8_0, ~6.9 GB, fits in fast memory, ~166 tok/s — ollama run d3:4b-q8_0

Setup on DGX Spark 128GB (NVIDIA, ASUS, Dell, Lenovo, HP, MSI, Acer, Gigabyte)

• Ollama: ollama run d3:4b-q8_0

Hardware that fits

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-23.

A project by OnDevice Labs · llms.txt