← CanIRunLocal · All hardware · All models

Can I run Nemotron 3.5 Bolt 74B-A9B (MoE, IQ2-class) locally?

NVIDIA · 74B parameters (9B active) · chat, reasoning, coding, vision.

Ollama weight files: Default 25 GB (nemotron-3.5-bolt:74b-a9b).

Yes on 35 machines in this list. The smallest is RTX 3090 24GB: Default, about 27.4 GB, roughly 32 tokens per second, via Ollama. ollama run nemotron-3.5-bolt:74b-a9b

DGX Spark 128GB: Default, ~27.4 GB, fits in fast memory, ~52 tok/s — ollama run nemotron-3.5-bolt:74b-a9b
Mac Studio M5 Max 128GB: Default, ~27.4 GB, fits in fast memory, ~81 tok/s — ollama run nemotron-3.5-bolt:74b-a9b
Mac Studio M5 Ultra 256GB: Default, ~27.4 GB, fits in fast memory, ~158 tok/s — ollama run nemotron-3.5-bolt:74b-a9b

Setup on DGX Spark 128GB (NVIDIA, ASUS, Dell, Lenovo, HP, MSI, Acer, Gigabyte)

Ollama: ollama run nemotron-3.5-bolt:74b-a9b
LM Studio: lms get unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF@Q8_0
llama.cpp: llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 -c 8192

Open Nemotron 3.5 Bolt 74B-A9B (MoE, IQ2-class) in the checker

Hardware that fits

Hardware	Runtime	Quant	Memory	Fit	Speed	Command
RTX 3090 24GB	Ollama	Default	27.4 GB	partial offload (slower)	~32 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
RTX 4090 24GB	Ollama	Default	27.4 GB	partial offload (slower)	~33 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
RX 7900 XTX 24GB	Ollama	Default	27.4 GB	partial offload (slower)	~33 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M3 Pro 36GB	Ollama	Default	27.4 GB	fits if the Mac GPU limit is raised	~20 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M4 Max (32c GPU) 36GB	Ollama	Default	27.4 GB	fits if the Mac GPU limit is raised	~54 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Max (32c GPU) 36GB	Ollama	Default	27.4 GB	fits if the Mac GPU limit is raised	~61 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
RTX 5090 32GB	Ollama	Default	27.4 GB	fits in fast memory	~177 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M4 Pro 48GB	Ollama	Default	27.4 GB	fits in fast memory	~36 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M4 Max (40c GPU) 48GB	Ollama	Default	27.4 GB	fits in fast memory	~72 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Max (40c GPU) 48GB	Ollama	Default	27.4 GB	fits in fast memory	~81 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M1 Max 64GB	Ollama	Default	27.4 GB	fits in fast memory	~53 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M2 Max 64GB	Ollama	Default	27.4 GB	fits in fast memory	~53 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M3 Max 64GB	Ollama	Default	27.4 GB	fits in fast memory	~53 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M4 Max 64GB	Ollama	Default	27.4 GB	fits in fast memory	~72 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M4 Max (40c GPU) 64GB	Ollama	Default	27.4 GB	fits in fast memory	~72 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Max (40c GPU) 64GB	Ollama	Default	27.4 GB	fits in fast memory	~81 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
DGX Spark 64GB (ASUS, Dell, HP, MSI, Acer, Gigabyte)	Ollama	Default	27.4 GB	fits in fast memory	~52 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M3 Ultra 96GB	Ollama	Default	27.4 GB	fits in fast memory	~108 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Ultra 96GB	Ollama	Default	27.4 GB	fits in fast memory	~158 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M3 Max 128GB	Ollama	Default	27.4 GB	fits in fast memory	~53 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M4 Max 128GB	Ollama	Default	27.4 GB	fits in fast memory	~72 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M4 Max (40c GPU) 128GB	Ollama	Default	27.4 GB	fits in fast memory	~72 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Max (40c GPU) 128GB	Ollama	Default	27.4 GB	fits in fast memory	~81 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
DGX Spark 128GB (NVIDIA, ASUS, Dell, Lenovo, HP, MSI, Acer, Gigabyte)	Ollama	Default	27.4 GB	fits in fast memory	~52 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
M2 Ultra 192GB	Ollama	Default	27.4 GB	fits in fast memory	~105 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
2x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~22 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra 256GB	Ollama	Default	27.4 GB	fits in fast memory	~108 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Ultra (64c GPU) 256GB	Ollama	Default	27.4 GB	fits in fast memory	~158 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
Mac Studio M5 Ultra (80c GPU) 256GB	Ollama	Default	27.4 GB	fits in fast memory	~158 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
2x DGX Spark 128GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~14 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~19 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
2x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~43 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra (80c GPU) 512GB	Ollama	Default	27.4 GB	fits in fast memory	~108 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
4x DGX Spark 128GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~12 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	81.2 GB	fits in fast memory	~38 tok/s	`llama-server -hf unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-16.

A project by OnDevice Labs · llms.txt