552B DeepSeek on M5 Ultra 256GB: 3883 Prefill, 92 Decode Tok/s – Local Full-Scale LLM Achieved

Benchmark Snapshot

On October 30, Beamsters announced MLX-Serve plugin support with a community-provided DeepSeek 4.1 STQ kernel. Two developers, STRML_ and ddalcu, ran deep optimizations on an M5 Ultra 256GB and reported:

Model: DeepSeek-V4.1-Flash-1017-iQ-MLX-3.0bpw — the October 17 open-source MIT-licensed 552B backbone (8B activated in, 16B out) quantized to 3 bits per weight in MLX format.

Engine: KV cache bf16, context 1024k (1 million tokens), MTP exact speculative mode + PLD (Prompt Lookup Decoding).

Test conditions: thinking enabled, greedy sampling, concurrency 1, warmed up once, median of 3 runs.

Concurrency 1 + greedy means this is real single-user speed under the most demanding scenario , not inflated by batch concurrency.

How 552B Fits in 256GB

Rough calculation: 552B parameters × 3 bits ÷ 8 ≈ 207 GB for weights. The M5 Ultra top configuration provides 256GB unified memory, leaving ~49GB for KV cache, OS, and inference framework — just enough, but with no margin.

This explains two design choices:

Why 3bpw not 4bpw: 4-bit version would need ~276GB, exceeding 256GB. 3-bit is lossy compression with some quality loss, but enables running the full 552B model instead of a truncated version.

Why KV cache stays bf16: The memory saved on weights must prioritize KV precision. A 1-million-token context consumes far more KV cache than expected; reducing KV precision hurts more than reducing weight precision.

Note: STQ is a community quantization kernel for DeepSeek 4.1; implementation details should be verified in the project repository.

Reading the Four Numbers

Four scenarios, each run 3 times taking median; data from tweet screenshot, not independently reproduced.

① Decode 92 tok/s — already a "readable" speed. Chinese reading speed ~5-8 characters/second; 92 tok/s is 10-18× reading speed. For comparison: M5 Ultra runs 70B dense Q4 at 15-25 tok/s, Qwen3.8 125B MoE at 154.5 tok/s — now a 552B model delivers 92 tok/s, fully exploiting MoE architectural advantage.

② Prefill 3883 tok/s — first-token latency nearly imperceptible. Feeding a 100k-token codebase or contract takes ~26 seconds. Combined with 1M-token context window, "feeding the entire project to a local model" becomes a practical daily workflow.

③ Speculative·predictable 155 tok/s (best run 163) — ~1.7× speedup. PLD (Prompt Lookup Decoding, a lookup-based speculative strategy) shows clear gains on "predictable content" tasks: rewriting, translation, code completion where context overlap is high. Speed approaches the Qwen 125B level — despite this model being 4.6× larger.

④ Speculative·novel 75 tok/s — slower than naive decoding. This is the most honest data point: the author didn't cherry-pick. On never-seen content, speculative hit rate drops, and 75 tok/s falls below the baseline 92 tok/s. Speculative decoding is not a free lunch; its benefit depends on how much your task and context overlap.

One-sentence reading: 92 is the floor, 3883 is the experience baseline, 155 is the ceiling, 75 is the honesty.

What This Means for You

Already own a 256GB M5 Ultra: Your machine just entered the "flagship open-source model full local inference" club. Once MLX-Serve plugin system stabilizes and the model file (ddalcu's 3.0bpw version) is released, you can reproduce the benchmark by following the card configuration.

Deciding between 256GB and 96GB: The 552B full-weight path is only viable on 256GB — 207GB weight footprint is non-negotiable. 96GB targets 125B-class MoE (Qwen3.8 measured at 154.5 tok/s), faster but with a lower capability ceiling.

Just watching: Understanding the test conditions (concurrency 1, greedy, median) matters more than the raw speeds. Next time you see "Device X runs Model Y at 500 tok/s", ask: under what conditions?

Data sourced from community tweets and benchmark screenshots; not independently reproduced. Calculations marked "estimated" are based on public parameters. But the direction is clear: when a 552B full model streams at 92 tok/s on a desktop, the statement "Mac can't run large models" can be officially retired.