← CanIRunLocal · All hardware · All models

What local LLMs run on M2 Ultra 128GB?

With 128 GB unified memory (about 96.0 GB usable by the GPU), these are the best open models that fit. The top pick is Qwen3.5 122B-A10B (MoE) at Q4_K_M (~83.7 GB, ~49 tok/s): ollama run qwen3.5:122b.

Best for coding: Qwen3.5 122B-A10B (MoE) (qwen3.5:122b), Nemotron 3.5 Super 120B-A12B (MoE) (nemotron-3.5-super:120b-a12b-q4_K_M), gpt-oss 120B (MoE, MXFP4) (gpt-oss:120b).

Runtimes available for this hardware: Ollama, LM Studio, llama.cpp, MLX.

Open the interactive checker for M2 Ultra 128GB

#	Model	Quant	Memory needed	Fit	Speed	Command
1	Qwen3.5 122B-A10B (MoE)	Q4_K_M	83.7 GB	fits in fast memory	~49 tok/s	`ollama run qwen3.5:122b`
2	Llama 4 Scout 109B (MoE)	Q4_K_M	69.7 GB	fits in fast memory	~31 tok/s	`ollama run llama4:16x17b`
3	Nemotron 3.5 Super 120B-A12B (MoE)	Q4_K_M	92.7 GB	fits in fast memory	~36 tok/s	`ollama run nemotron-3.5-super:120b-a12b-q4_K_M`
4	gpt-oss 120B (MoE, MXFP4)	MXFP4	67.3 GB	fits in fast memory	~113 tok/s	`ollama run gpt-oss:120b`
5	Nemotron 3.5 Bolt 74B-A9B (MoE, IQ2-class)	Default	27.7 GB	fits in fast memory	~105 tok/s	`ollama run nemotron-3.5-bolt:74b-a9b`
6	Qwen3-Next 80B-A3B (MoE)	Q8_0	87.3 GB	fits in fast memory	~100 tok/s	`ollama run qwen3-next:80b-a3b-instruct-q8_0`
7	Bonsai 2 27B (ternary)	Q2_0	10.6 GB	fits in fast memory	~77 tok/s	`ollama run bonsai-2:27b`
8	Qwen3.5 35B-A3B (MoE)	Q8_0	40.9 GB	fits in fast memory	~98 tok/s	`ollama run qwen3.5:35b-a3b-q8_0`
9	Qwen3.6 35B-A3B (MoE)	Q8_0	40.9 GB	fits in fast memory	~98 tok/s	`ollama run qwen3.6:35b-a3b-q8_0`
10	Qwen3 30B-A3B (MoE)	Q8_0	34.4 GB	fits in fast memory	~90 tok/s	`ollama run qwen3:30b-a3b-q8_0`
11	Qwen3-Coder 30B-A3B (MoE)	Q8_0	33.4 GB	fits in fast memory	~92 tok/s	`ollama run qwen3-coder:30b-a3b-q8_0`
12	Laguna XS 2.1 33B-A3B (MoE)	Q8_0	37.4 GB	fits in fast memory	~99 tok/s	`ollama run laguna-xs-2.1:q8_0`
13	Gemma 4 26B-A4B (MoE)	Q8_0	29.6 GB	fits in fast memory	~78 tok/s	`ollama run gemma4:26b-a4b-it-q8_0`
14	Nemotron 3.5 Lightning 30B-A3B (MoE)	Q8_0	36.4 GB	fits in fast memory	~96 tok/s	`ollama run nemotron-3.5-lightning:30b-a3b-q8_0`
15	GLM-4.7-Flash 30B-A3B (MoE)	Q8_0	33.4 GB	fits in fast memory	~104 tok/s	`ollama run glm-4.7-flash:q8_0`
16	Qwen3-VL 30B-A3B (MoE)	Q8_0	35.7 GB	fits in fast memory	~98 tok/s	`ollama run qwen3-vl:30b-a3b-instruct-q8_0`
17	Ornith-1.5 35B-A3B (MoE)	Default	24.8 GB	fits in fast memory	~162 tok/s	`ollama run ornith-1.5:35b`
18	Llama 3.3 70B	Q8_0	79.1 GB	fits in fast memory	~9 tok/s	`ollama run llama3.3:70b-instruct-q8_0`
19	DeepSeek R1 Distill Llama 70B	Q8_0	79.1 GB	fits in fast memory	~9 tok/s	`ollama run deepseek-r1:70b-llama-distill-q8_0`
20	gpt-oss 20B (MoE, MXFP4)	MXFP4	15.1 GB	fits in fast memory	~133 tok/s	`ollama run gpt-oss:20b`
21	Qwen3-VL 32B	Q8_0	38.6 GB	fits in fast memory	~18 tok/s	`ollama run qwen3-vl:32b-instruct-q8_0`
22	Qwen3 32B	Q8_0	37.2 GB	fits in fast memory	~18 tok/s	`ollama run qwen3:32b-q8_0`
23	DeepSeek R1 Distill Qwen 32B	Q8_0	37.2 GB	fits in fast memory	~18 tok/s	`ollama run deepseek-r1:32b-qwen-distill-q8_0`
24	North Mini Code 1.0 30B (MoE)	Q8_0	34.1 GB	fits in fast memory	~20 tok/s	`ollama run north-mini-code-1.0:q8_0`
25	LFM2.5 24B-A2B (MoE)	Q8_0	26.2 GB	fits in fast memory	~154 tok/s	`ollama run lfm2.5:24b-a2b-q8_0`
26	Muse Glimmer 30B	Q8_0	33.4 GB	fits in fast memory	~21 tok/s	`ollama run muse-glimmer:30b-q8_0`
27	Gemma 4 31B	Q8_0	36.5 GB	fits in fast memory	~19 tok/s	`ollama run gemma4:31b-it-q8_0`
28	Granite 4.2 30B	Q8_0	33.1 GB	fits in fast memory	~21 tok/s	`ollama run granite4.2:30b-q8_0`
29	Qwen3.5 27B	Q8_0	32.3 GB	fits in fast memory	~21 tok/s	`ollama run qwen3.5:27b-q8_0`
30	Qwen3.6 27B	Q8_0	32.3 GB	fits in fast memory	~21 tok/s	`ollama run qwen3.6:27b-q8_0`
31	Gemma 3 27B	Q8_0	32.3 GB	fits in fast memory	~21 tok/s	`ollama run gemma3:27b-it-q8_0`
32	Clef 27B (Cloudflare, decision)	Q8_0	32.3 GB	fits in fast memory	~21 tok/s	`ollama run clef:27b-q8_0`
33	Qwen3.8 27B	Q8_0	32.3 GB	fits in fast memory	~21 tok/s	`ollama run qwen3.8:27b-q8_0`
34	Mistral Small 24B	Q8_0	26.8 GB	fits in fast memory	~26 tok/s	`ollama run mistral-small:24b-instruct-2501-q8_0`
35	Devstral Small 2 24B	Q8_0	27.8 GB	fits in fast memory	~25 tok/s	`ollama run devstral-small-2:24b-instruct-2512-q8_0`
36	Qwen3 14B	Q8_0	17.3 GB	fits in fast memory	~40 tok/s	`ollama run qwen3:14b-q8_0`
37	DeepSeek R1 Distill Qwen 14B	Q8_0	17.3 GB	fits in fast memory	~40 tok/s	`ollama run deepseek-r1:14b-qwen-distill-q8_0`
38	Phi-4 14B	Q8_0	17.3 GB	fits in fast memory	~40 tok/s	`ollama run phi4:14b-q8_0`
39	Ministral 3 14B	Q8_0	16.6 GB	fits in fast memory	~43 tok/s	`ollama run ministral-3:14b-instruct-2512-q8_0`
40	Mixtral 8x7B (MoE)	Q8_0	51.8 GB	fits in fast memory	~23 tok/s	`ollama run mixtral:8x7b-instruct-v0.1-q8_0`
41	Mistral Nemo 12B	Q8_0	14.2 GB	fits in fast memory	~49 tok/s	`ollama run mistral-nemo:12b-instruct-2407-q8_0`
42	Gemma 3 12B	Q8_0	14.6 GB	fits in fast memory	~49 tok/s	`ollama run gemma3:12b-it-q8_0`
43	Gemma 4 12B	Q8_0	14.6 GB	fits in fast memory	~49 tok/s	`ollama run gemma4:12b-it-q8_0`
44	Llama 3.2 Vision 11B	Q8_0	13.5 GB	fits in fast memory	~53 tok/s	`ollama run llama3.2-vision:11b-instruct-q8_0`
45	Qwen3.5 9B	Q8_0	11.4 GB	fits in fast memory	~64 tok/s	`ollama run qwen3.5:9b-q8_0`
46	Nimble v3 9B (Bespoke Labs, decision)	Q8_0	10.6 GB	fits in fast memory	~67 tok/s	`ollama run nimble-v3:9b-q8_0`
47	Ministral 3 8B	Q8_0	11.3 GB	fits in fast memory	~65 tok/s	`ollama run ministral-3:8b-instruct-2512-q8_0`
48	Clef-Flash 9B (Cloudflare, decision)	Q8_0	12.4 GB	fits in fast memory	~58 tok/s	`ollama run clef-flash:9b-q8_0`
49	Granite 4.2 8B	Q8_0	10.3 GB	fits in fast memory	~69 tok/s	`ollama run granite4.2:8b-q8_0`
50	Qwen3-VL 8B	Q8_0	11.2 GB	fits in fast memory	~65 tok/s	`ollama run qwen3-vl:8b-instruct-q8_0`
51	Qwen 2.5 VL 7B	Q8_0	10.8 GB	fits in fast memory	~68 tok/s	`ollama run qwen2.5vl:7b-q8_0`
52	Qwen3 8B	Q8_0	9.9 GB	fits in fast memory	~72 tok/s	`ollama run qwen3:8b-q8_0`
53	DeepSeek R1 0528 Qwen3 8B	Q8_0	9.9 GB	fits in fast memory	~72 tok/s	`ollama run deepseek-r1:8b-0528-qwen3-q8_0`
54	MiniCPM-V 2.6 8B	Q8_0	10.5 GB	fits in fast memory	~70 tok/s	`ollama run minicpm-v:8b-2.6-q8_0`
55	Gemma 3n E4B (on-device)	Q8_0	10.5 GB	fits in fast memory	~67 tok/s	`ollama run gemma3n:e4b-it-q8_0`
56	Gemma 4 E4B (effective 4.5B)	Q8_0	10.3 GB	fits in fast memory	~71 tok/s	`ollama run gemma4:e4b-it-q8_0`
57	Bonsai 2 9B (ternary)	Q2_0	4.6 GB	fits in fast memory	~200 tok/s	`ollama run bonsai-2:9b`
58	DeepSeek R1 Distill Qwen 7B	Q8_0	9.1 GB	fits in fast memory	~79 tok/s	`ollama run deepseek-r1:7b-qwen-distill-q8_0`
59	Ornith-1.5 9B	Default	8 GB	fits in fast memory	~97 tok/s	`ollama run ornith-1.5:9b`
60	LFM2.5 8B-A1B (MoE)	Q8_0	9.8 GB	fits in fast memory	~301 tok/s	`ollama run lfm2.5:8b-a1b-q8_0`
61	Qwen 2.5 72B	Q8_0	81.1 GB	fits in fast memory	~8 tok/s	`ollama run qwen2.5:72b-instruct-q8_0`
62	Gemma 3n E2B (on-device)	Q8_0	7.8 GB	fits in fast memory	~93 tok/s	`ollama run gemma3n:e2b-it-q8_0`
63	Qwen 2.5 32B	Q8_0	37.2 GB	fits in fast memory	~18 tok/s	`ollama run qwen2.5:32b-instruct-q8_0`
64	Qwen 2.5 Coder 32B	Q8_0	37.2 GB	fits in fast memory	~18 tok/s	`ollama run qwen2.5-coder:32b-instruct-q8_0`
65	Gemma 4 E2B (effective 2.3B)	Q8_0	7.2 GB	fits in fast memory	~107 tok/s	`ollama run gemma4:e2b-it-q8_0`
66	Qwen3.5 4B	Q8_0	6.4 GB	fits in fast memory	~123 tok/s	`ollama run qwen3.5:4b-q8_0`
67	DeepSeek Coder V2 Lite 16B (MoE)	Q8_0	18 GB	fits in fast memory	~123 tok/s	`ollama run deepseek-coder-v2:16b-lite-instruct-q8_0`
68	Qwen3-VL 4B	Q8_0	6.3 GB	fits in fast memory	~125 tok/s	`ollama run qwen3-vl:4b-instruct-q8_0`
69	Gemma 3 4B	Q8_0	6.2 GB	fits in fast memory	~128 tok/s	`ollama run gemma3:4b-it-q8_0`
70	Qwen3 4B	Q8_0	5.2 GB	fits in fast memory	~145 tok/s	`ollama run qwen3:4b-q8_0`
71	tev1 4B (Together AI, decision)	Q8_0	5.4 GB	fits in fast memory	~139 tok/s	`ollama run tev1:4b-q8_0`
72	Ministral 3 3B	Q8_0	5.6 GB	fits in fast memory	~142 tok/s	`ollama run ministral-3:3b-instruct-2512-q8_0`
73	Phi-4 Mini 3.8B	Q8_0	4.9 GB	fits in fast memory	~156 tok/s	`ollama run phi4-mini:3.8b-q8_0`
74	Qwen 2.5 14B	Q8_0	17.3 GB	fits in fast memory	~40 tok/s	`ollama run qwen2.5:14b-instruct-q8_0`
75	Qwen 2.5 Coder 14B	Q8_0	17.3 GB	fits in fast memory	~40 tok/s	`ollama run qwen2.5-coder:14b-instruct-q8_0`
76	Granite 4.2 3B	Q8_0	4.7 GB	fits in fast memory	~164 tok/s	`ollama run granite4.2:3b-q8_0`
77	Bonsai 2 4B (ternary)	Q2_0	2.8 GB	fits in fast memory	~400 tok/s	`ollama run bonsai-2:4b`
78	Llama 3.2 3B	Q8_0	4.2 GB	fits in fast memory	~188 tok/s	`ollama run llama3.2:3b-instruct-q8_0`
79	Llama 3.1 8B	Q8_0	9.5 GB	fits in fast memory	~75 tok/s	`ollama run llama3.1:8b-instruct-q8_0`
80	Qwen3.5 2B	Q8_0	3.8 GB	fits in fast memory	~237 tok/s	`ollama run qwen3.5:2b-q8_0`
81	Qwen 2.5 7B	Q8_0	9.1 GB	fits in fast memory	~79 tok/s	`ollama run qwen2.5:7b-instruct-q8_0`
82	Qwen 2.5 Coder 7B	Q8_0	9.1 GB	fits in fast memory	~79 tok/s	`ollama run qwen2.5-coder:7b-instruct-q8_0`
83	Qwen3-VL 2B	Q8_0	3.8 GB	fits in fast memory	~237 tok/s	`ollama run qwen3-vl:2b-instruct-q8_0`
84	Mistral 7B v0.3	Q8_0	8.7 GB	fits in fast memory	~83 tok/s	`ollama run mistral:7b-instruct-q8_0`
85	LLaVA 1.6 7B	Q8_0	9.1 GB	fits in fast memory	~82 tok/s	`ollama run llava:7b-v1.6-vicuna-q8_0`
86	Qwen3 1.7B	Q8_0	2.9 GB	fits in fast memory	~291 tok/s	`ollama run qwen3:1.7b-q8_0`
87	Code Llama 7B	Q8_0	8.1 GB	fits in fast memory	~89 tok/s	`ollama run codellama:7b-instruct-q8_0`
88	DeepSeek R1 Distill Qwen 1.5B	Q8_0	2.6 GB	fits in fast memory	~337 tok/s	`ollama run deepseek-r1:1.5b-qwen-distill-q8_0`
89	SmolLM2 1.7B	Q8_0	2.5 GB	fits in fast memory	~356 tok/s	`ollama run smollm2:1.7b`
90	Bonsai 2 2B (ternary)	Q2_0	2 GB	fits in fast memory	~703 tok/s	`ollama run bonsai-2:2b`
91	GLM-OCR 0.9B	BF16	3.7 GB	fits in fast memory	~237 tok/s	`ollama run glm-ocr:latest`
92	MiniCPM-V 4.6 1B	Q8_0	2.9 GB	fits in fast memory	~337 tok/s	`ollama run minicpm-v4.6:q8_0`
93	Llama 3.2 1B	Q8_0	2 GB	fits in fast memory	~492 tok/s	`ollama run llama3.2:1b`
94	LFM2.5 1.2B Thinking	Q8_0	1.9 GB	fits in fast memory	~533 tok/s	`ollama run lfm2.5-thinking:1.2b-q8_0`
95	Gemma 3 1B	Q8_0	1.8 GB	fits in fast memory	~582 tok/s	`ollama run gemma3:1b-it-q8_0`
96	Qwen3.5 0.8B	Q8_0	2 GB	fits in fast memory	~640 tok/s	`ollama run qwen3.5:0.8b-q8_0`
97	Qwen3 0.6B	Q8_0	1.5 GB	fits in fast memory	~771 tok/s	`ollama run qwen3:0.6b-q8_0`
98	Qwen 2.5 3B	Q8_0	4.1 GB	fits in fast memory	~194 tok/s	`ollama run qwen2.5:3b-instruct-q8_0`
99	SmolLM2 360M	FP16	1.3 GB	fits in fast memory	~877 tok/s	`ollama run smollm2:360m`
100	SmolLM2 135M	FP16	0.9 GB	fits in fast memory	~2370 tok/s	`ollama run smollm2:135m`
101	Qwen 2.5 Coder 1.5B	Q8_0	2.3 GB	fits in fast memory	~400 tok/s	`ollama run qwen2.5-coder:1.5b-instruct-q8_0`

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-18.

A project by OnDevice Labs · llms.txt