What command runs Llama 4 Maverick 400B (MoE)?

vllm serve RedHatAI/Llama-4-Maverick-17B-128E-Instruct-quantized.w4a16 --tensor-parallel-size 2 --distributed-executor-backend ray --max-model-len 8192

← CanIRunLocal · All hardware · All models

Can I run Llama 4 Maverick 400B (MoE) locally?

Meta · 400B parameters (17B active) · chat, vision.

Ollama weight files: Q4_K_M 245 GB (llama4:128x17b).

Yes on 9 machines in this list. The smallest is Mac Studio M3 Ultra 256GB: Q4_K_M, about 228.3 GB, roughly 34 tokens per second, via LM Studio. lms get mlx-community/Llama-4-Maverick-17B-128E-Instruct-4bit

DGX Spark 128GB: does not fit
Mac Studio M5 Max 128GB: does not fit
Mac Studio M5 Ultra 256GB: Q4_K_M, ~228.3 GB, fits if the Mac GPU limit is raised, ~50 tok/s — lms get mlx-community/Llama-4-Maverick-17B-128E-Instruct-4bit

Setup on 2x DGX Spark 128GB

vLLM: vllm serve RedHatAI/Llama-4-Maverick-17B-128E-Instruct-quantized.w4a16 --tensor-parallel-size 2 --distributed-executor-backend ray --max-model-len 8192

Open Llama 4 Maverick 400B (MoE) in the checker

Hardware that fits

Hardware	Runtime	Quant	Memory	Fit	Speed	Command
Mac Studio M3 Ultra 256GB	LM Studio	MLX 4bit	228.3 GB	fits if the Mac GPU limit is raised	~34 tok/s	`lms get mlx-community/Llama-4-Maverick-17B-128E-Instruct-4bit`
Mac Studio M5 Ultra (64c GPU) 256GB	LM Studio	MLX 4bit	228.3 GB	fits if the Mac GPU limit is raised	~50 tok/s	`lms get mlx-community/Llama-4-Maverick-17B-128E-Instruct-4bit`
Mac Studio M5 Ultra (80c GPU) 256GB	LM Studio	MLX 4bit	228.3 GB	fits if the Mac GPU limit is raised	~50 tok/s	`lms get mlx-community/Llama-4-Maverick-17B-128E-Instruct-4bit`
2x DGX Spark 128GB	vLLM	W4A16	218.3 GB	fits in fast memory	~21 tok/s	`vllm serve RedHatAI/Llama-4-Maverick-17B-128E-Instruct-quantized.w4a16 --tensor-parallel-size 2 --distributed-executor-backend ray --max-model-len 8192`
4x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q5_K_M GGUF	286.8 GB	fits in fast memory	~15 tok/s	`llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q5_K_M --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
2x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q5_K_M GGUF	286.8 GB	fits in fast memory	~34 tok/s	`llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q5_K_M --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra (80c GPU) 512GB	Ollama	Q4_K_M	247.3 GB	fits in fast memory	~31 tok/s	`ollama run llama4:128x17b`
4x DGX Spark 128GB	llama.cpp	Q8_0 GGUF	428.2 GB	fits in fast memory	~7 tok/s	`llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	428.2 GB	fits in fast memory	~20 tok/s	`llama-server -hf unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-16.

A project by OnDevice Labs · llms.txt