← CanIRunLocal · All hardware · All models

Can I run Mixtral 8x7B (MoE) locally?

Mistral · 46.7B parameters (12.9B active) · chat.

Ollama weight files: Q5_K_M 33 GB (mixtral:8x7b-instruct-v0.1-q5_K_M); Q8_0 50 GB (mixtral:8x7b-instruct-v0.1-q8_0); Q4_0 26 GB (mixtral:8x7b); Q4_K_M 28 GB (mixtral:8x7b-instruct-v0.1-q4_K_M).

Yes on 35 machines in this list. The smallest is RTX 3090 24GB: Q4_K_M, about 29.8 GB, roughly 10 tokens per second, via Ollama. ollama run mixtral:8x7b-instruct-v0.1-q4_K_M

Setup on DGX Spark 128GB (NVIDIA, ASUS, Dell, Lenovo, HP, MSI, Acer, Gigabyte)

Open Mixtral 8x7B (MoE) in the checker

Hardware that fits

HardwareRuntimeQuantMemoryFitSpeedCommand
RTX 3090 24GBOllamaQ4_K_M29.8 GBpartial offload (slower)~10 tok/sollama run mixtral:8x7b-instruct-v0.1-q4_K_M
RTX 4090 24GBOllamaQ4_K_M29.8 GBpartial offload (slower)~10 tok/sollama run mixtral:8x7b-instruct-v0.1-q4_K_M
RX 7900 XTX 24GBOllamaQ4_K_M29.8 GBpartial offload (slower)~10 tok/sollama run mixtral:8x7b-instruct-v0.1-q4_K_M
M3 Pro 36GBLM StudioMLX 4bit28 GBfits if the Mac GPU limit is raised~8 tok/slms get mlx-community/Mixtral-8x7B-Instruct-v0.1-4bit
Mac Studio M4 Max (32c GPU) 36GBLM StudioMLX 4bit28 GBfits if the Mac GPU limit is raised~23 tok/slms get mlx-community/Mixtral-8x7B-Instruct-v0.1-4bit
Mac Studio M5 Max (32c GPU) 36GBLM StudioMLX 4bit28 GBfits if the Mac GPU limit is raised~25 tok/slms get mlx-community/Mixtral-8x7B-Instruct-v0.1-4bit
RTX 5090 32GBOllamaQ4_K_M29.8 GBfits in fast memory~70 tok/sollama run mixtral:8x7b-instruct-v0.1-q4_K_M
M4 Pro 48GBOllamaQ5_K_M34.8 GBfits in fast memory~12 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
Mac Studio M4 Max (40c GPU) 48GBOllamaQ5_K_M34.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
Mac Studio M5 Max (40c GPU) 48GBOllamaQ5_K_M34.8 GBfits in fast memory~27 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
M1 Max 64GBOllamaQ5_K_M34.8 GBfits in fast memory~18 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
M2 Max 64GBOllamaQ5_K_M34.8 GBfits in fast memory~18 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
M3 Max 64GBOllamaQ5_K_M34.8 GBfits in fast memory~18 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
M4 Max 64GBOllamaQ5_K_M34.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
Mac Studio M4 Max (40c GPU) 64GBOllamaQ5_K_M34.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
Mac Studio M5 Max (40c GPU) 64GBOllamaQ5_K_M34.8 GBfits in fast memory~27 tok/sollama run mixtral:8x7b-instruct-v0.1-q5_K_M
DGX Spark 64GB (ASUS, Dell, HP, MSI, Acer, Gigabyte)OllamaQ8_051.8 GBfits in fast memory~11 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M3 Ultra 96GBOllamaQ8_051.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M5 Ultra 96GBOllamaQ8_051.8 GBfits in fast memory~35 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
M3 Max 128GBOllamaQ8_051.8 GBfits in fast memory~12 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
M4 Max 128GBOllamaQ8_051.8 GBfits in fast memory~16 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M4 Max (40c GPU) 128GBOllamaQ8_051.8 GBfits in fast memory~16 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M5 Max (40c GPU) 128GBOllamaQ8_051.8 GBfits in fast memory~18 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
DGX Spark 128GB (NVIDIA, ASUS, Dell, Lenovo, HP, MSI, Acer, Gigabyte)OllamaQ8_051.8 GBfits in fast memory~11 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
M2 Ultra 192GBOllamaQ8_051.8 GBfits in fast memory~23 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
2x Mac Studio M5 Max (40c GPU) 128GBMLXMLX 8bit51.4 GBfits in fast memory~25 tok/sexo # on every Mac (brew install --cask exo); load mlx-community/Mixtral-8x7B-Instruct-v0.1-8bit with Tensor sharding + RDMA
Mac Studio M3 Ultra 256GBOllamaQ8_051.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M5 Ultra (64c GPU) 256GBOllamaQ8_051.8 GBfits in fast memory~35 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
Mac Studio M5 Ultra (80c GPU) 256GBOllamaQ8_051.8 GBfits in fast memory~35 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
2x DGX Spark 128GBvLLMOfficial weights95.2 GBfits in fast memory~8 tok/svllm serve mistralai/Mixtral-8x7B-Instruct-v0.1 --tensor-parallel-size 2 --distributed-executor-backend ray --max-model-len 8192
4x Mac Studio M5 Max (40c GPU) 128GBMLXMLX 8bit51.4 GBfits in fast memory~39 tok/sexo # on every Mac (brew install --cask exo); load mlx-community/Mixtral-8x7B-Instruct-v0.1-8bit with Tensor sharding + RDMA
2x Mac Studio M5 Ultra (80c GPU) 256GBMLXMLX 8bit51.4 GBfits in fast memory~49 tok/sexo # on every Mac (brew install --cask exo); load mlx-community/Mixtral-8x7B-Instruct-v0.1-8bit with Tensor sharding + RDMA
Mac Studio M3 Ultra (80c GPU) 512GBOllamaQ8_051.8 GBfits in fast memory~24 tok/sollama run mixtral:8x7b-instruct-v0.1-q8_0
4x DGX Spark 128GBvLLMOfficial weights95.2 GBfits in fast memory~11 tok/svllm serve mistralai/Mixtral-8x7B-Instruct-v0.1 --tensor-parallel-size 4 --distributed-executor-backend ray --max-model-len 8192
4x Mac Studio M5 Ultra (80c GPU) 256GBMLXMLX 8bit51.4 GBfits in fast memory~77 tok/sexo # on every Mac (brew install --cask exo); load mlx-community/Mixtral-8x7B-Instruct-v0.1-8bit with Tensor sharding + RDMA

Estimates, not benchmarks. Model sizes are ollama.com download sizes; memory needed adds headroom for ~8k tokens of context; speeds are rough figures from memory bandwidth. Data updated 2026-10-16.

A project by OnDevice Labs · llms.txt