Local LLM Hardware Truths: MoE, CPU/GPU/NPU, and 32GB Mac Mini Tuning
This article explains why CPUs struggle with MoE model prefill but excel at decode, compares dense, SSM, MLA, and BitNet architectures, evaluates CPU, NVIDIA GPU, Apple UMA, and NPU hardware for local LLM inference, and provides a practical tuning guide for running 32B models on a 32GB Mac mini with MLX and Ollama.
