← CanIRunLocal · All hardware · All models

Can I run Ling-3.1-flash 560B-A25B (MoE) locally?

inclusionAI · 560B parameters (25B active) · chat, reasoning, coding.

Ollama weight files: Q4_0 326.64 GB (hf.co/bartowski/Ling-3.1-flash-GGUF:Q4_0, size from the Hugging Face file listing); Q4_K_M 357.97 GB (hf.co/bartowski/Ling-3.1-flash-GGUF:Q4_K_M, size from the Hugging Face file listing); Q5_K_M 436.32 GB (hf.co/bartowski/Ling-3.1-flash-GGUF:Q5_K_M, size from the Hugging Face file listing); Q8_0 608.38 GB (hf.co/bartowski/Ling-3.1-flash-GGUF:Q8_0, size from the Hugging Face file listing).

Yes on 5 machines in this list. The smallest is 4x Mac Studio M5 Max (40c GPU) 128GB: Q4_K_M, about 360.3 GB, roughly 12 tokens per second, via llama.cpp. llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q4_K_M --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node

DGX Spark 128GB: does not fit
Mac Studio M5 Max 128GB: does not fit
Mac Studio M5 Ultra 256GB: does not fit

Setup on 4x Mac Studio M5 Max (40c GPU) 128GB

llama.cpp: llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q4_K_M --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node

Open Ling-3.1-flash 560B-A25B (MoE) in the checker

Hardware that fits

Hardware	Runtime	Quant	Memory	Fit	Speed	Command
4x Mac Studio M5 Max (40c GPU) 128GB	llama.cpp	Q4_K_M GGUF	360.3 GB	fits in fast memory	~12 tok/s	`llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q4_K_M --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
2x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q4_K_M GGUF	360.3 GB	fits in fast memory	~26 tok/s	`llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q4_K_M --rpc node2:50052 -c 8192 # run rpc-server on each other node`
Mac Studio M3 Ultra (80c GPU) 512GB	Ollama	Q4_K_M	360.3 GB	fits in fast memory	~20 tok/s	`ollama run hf.co/bartowski/Ling-3.1-flash-GGUF:Q4_K_M`
4x DGX Spark 128GB	llama.cpp	Q4_K_M GGUF	360.3 GB	fits in fast memory	~7 tok/s	`llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q4_K_M --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`
4x Mac Studio M5 Ultra (80c GPU) 256GB	llama.cpp	Q8_0 GGUF	610.7 GB	fits in fast memory	~13 tok/s	`llama-server -hf bartowski/Ling-3.1-flash-GGUF:Q8_0 --rpc node2:50052,node3:50052,node4:50052 -c 8192 # run rpc-server on each other node`

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-20.

A project by OnDevice Labs · llms.txt