M5 Max 128GB: Running 125B Qwen3.8-Flash-Next at 60 tok/s with MLX & MTP

Hardware Verification: M5 Max 128GB Bandwidth Is 614GB/s

The M5 Max comes in two GPU variants: the standard 32-core GPU with 460GB/s bandwidth, and the 128GB memory configuration which must include the 40-core GPU delivering 614GB/s bandwidth (per Apple support page support.apple.com/zh-cn/128107, 2026 model). This 614GB/s ceiling is a hard wall that caps decode throughput; all optimizations merely operate under this limit.

Model Format Pitfalls: Original NVFP4 Won't Run on Mac

The upstream Qwen3.8-Flash-Next release uses NVIDIA NVFP4 format (~96 GiB), exclusive to Blackwell GPUs. Apple GPUs cannot load it. On Mac you must use community-converted MLX or GGUF versions.

Another hidden trap: the model is not literally 125B parameters. It comprises 125B expert weights + 51B n-gram embedding table + 4B MTP layers, totalling ~180B resident parameters. The n-gram table is typically kept near full precision, so a naive "125B at 4-bit fits in 60GB" estimate is wrong. Actual 4-bit sizes are shown in the next section.

Memory Accounting on 128GB (Critical)

macOS hidden limit (must fix): By default the OS allocates only ~70–75% of physical RAM to the GPU (~96GB on 128GB). Without raising this ceiling the model will OOM on load. One command raises the limit without reboot or pre-allocation:

sudo sysctl iogpu.wired_limit_mb=114688   # ≈112GB ceiling

After this, the IQ4_NL quantization (102GB) fits with ~10GB headroom for KV cache and system. For tighter memory, the 51B n-gram table can be offloaded to SSD (covered later).

Framework Selection (Ranked by Speed)

1st choice: MLX ecosystem with MTP (speculative decoding) enabled — fastest.

OMLX (jundot/omlx): Native MTP speculative decoding + SSD engram offload + paged KV cache. Community tests on M5 Max show single-stream decode ~44–60 tok/s with MTP on.

MLX-Serve (ddalcu/mlx-serve): Zig-based, extremely lightweight. Author's benchmark anchor is M5 Ultra 256GB hitting 154.5 tok/s on 125B MoE; no direct M5 Max measurement exists , so do not extrapolate. mlx-vlm (v0.7+) also works; upstream mlx-lm does not yet support the architecture (PR still open).

2nd choice: llama.cpp (GGUF). IQ4_NL plain decode ~30 tok/s (MTP off). Advantages: broadest compatibility, stronger long-context handling (40K+ tokens where MLX advantage narrows or reverses), multimodal via mmproj. Requires llama.cpp built after 2026-08-27 for qwen4_exp architecture support, otherwise "unknown architecture" error.

Why MLX is markedly faster on MoE: MLX outperforms llama.cpp by 1.4–3× on MoE models. The M5 Neural Accelerator makes MLX's time-to-first-token ~4× faster than llama.cpp's Metal backend (prefill rises from ~1150 to ~1810 tok/s).

Command-Level Optimization Checklist

sudo sysctl iogpu.wired_limit_mb=114688
llama-server -hf ggml-org/Qwen3.8-Flash-Next-GGUF:IQ4_NL \
-ngl 99 -c 32768 --jinja -fit off \
--host 127.0.0.1 --port 8080

Three acceleration levers, ranked by gain: ① Enable MTP speculative decoding (biggest jump, 30→60 tok/s) > ② Use MLX instead of llama.cpp (1.4–3× on MoE) > ③ Offload n-gram table to SSD (saves memory, reduces load pressure, but slows prefill).

Realistic Performance Expectations (Community Aggregates, Not Official Benchmarks)

On M5 Max 128GB the practical range is: with MTP ~45–60 tok/s, without MTP ~30 tok/s . Compared to M5 Ultra's 57–154 tok/s, the gap stems from both bandwidth (614 vs 1200 GB/s) and GPU cores (40 vs 80) being roughly half — a physical limit software cannot overcome.

Pitfall Checklist

❌ Skipping iogpu.wired_limit adjustment → immediate OOM on load.

❌ Downloading original NVFP4 (targeted at RTX 6000 / DGX Spark).

❌ Attempting Q8_0 (163GB, impossible on 128GB).

❌ Using old llama.cpp lacking qwen4_exp support.

⚠️ Commercial use: Qwen Community 1.0 permits internal/local use freely; external MaaS or coding products require separate license from Alibaba.

Data sources: Apple official specs (hardware), LLMCheck community aggregate (2026-10-18, per-chip per-framework tok/s), daily.dev (M5 Max MLX vs llama.cpp benchmarks), presenc.ai (M5 Neural Accelerator baselines), AtomicChat deployment guide (SSD offload & -fit off usage). All performance figures are community aggregates, not a unified official benchmark; actual speed varies with thermals, context length, and quantization format.