← CanIRunLocal · All hardware · All models

Can I run Qwen3-VL 235B-A22B (MoE) locally?

Alibaba · 236B parameters (22B active) · vision, chat, reasoning.

Ollama weight files: Q4_K_M 143 GB (qwen3-vl:235b); Q8_0 251 GB (qwen3-vl:235b-a22b-instruct-q8_0).

Yes on 11 machines in this list. The smallest is M2 Ultra 192GB: Q4_K_M, about 136.1 GB, roughly 26 tokens per second, via LM Studio. lms get mlx-community/Qwen3-VL-235B-A22B-Instruct-4bit

DGX Spark 128GB: does not fit
Mac Studio M5 Max 128GB: does not fit
Mac Studio M5 Ultra 256GB: Q4_K_M, ~145.7 GB, fits in fast memory, ~36 tok/s — ollama run qwen3-vl:235b

Setup on Mac Studio M5 Ultra (80c GPU) 256GB

Ollama: ollama run qwen3-vl:235b
LM Studio: lms get mlx-community/Qwen3-VL-235B-A22B-Instruct-4bit
llama.cpp: llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q5_K_M -c 8192
MLX: mlx_lm.generate --model mlx-community/Qwen3-VL-235B-A22B-Instruct-4bit --prompt "Hello"

Open Qwen3-VL 235B-A22B (MoE) in the checker

Hardware that fits

Hardware	Runtime	Quant	Memory	Fit	Speed	Command
M2 Ultra 192GB	LM Studio	MLX 4bit	136.1 GB	fits in fast memory	~26 tok/s	`lms get mlx-community/Qwen3-VL-235B-A22B-Instruct-4bit`
2x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q5_K_M GGUF	169.5 GB	fits in fast memory	~13 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q5_K_M --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra 256GB	Ollama	Q4_K_M	145.7 GB	fits in fast memory	~25 tok/s	`ollama run qwen3-vl:235b`
Mac Studio M5 Ultra (64c GPU) 256GB	Ollama	Q4_K_M	145.7 GB	fits in fast memory	~36 tok/s	`ollama run qwen3-vl:235b`
Mac Studio M5 Ultra (80c GPU) 256GB	Ollama	Q4_K_M	145.7 GB	fits in fast memory	~36 tok/s	`ollama run qwen3-vl:235b`
2x DGX Spark 128GB	llama.cpp	Q5_K_M GGUF	169.5 GB	fits in fast memory	~9 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q5_K_M --rpc node2:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q8_0 GGUF	252.6 GB	fits in fast memory	~8 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
2x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	252.6 GB	fits in fast memory	~18 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q8_0 --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra (80c GPU) 512GB	Ollama	Q8_0	253.7 GB	fits in fast memory	~14 tok/s	`ollama run qwen3-vl:235b-a22b-instruct-q8_0`
4x DGX Spark 128GB	llama.cpp	Q8_0 GGUF	252.6 GB	fits in fast memory	~5 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	252.6 GB	fits in fast memory	~15 tok/s	`llama-server -hf unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-18.

A project by OnDevice Labs · llms.txt