61.9 tok/s on Mac mini M6: 6.1× Speculative Decoding Beats 170GB/s Bandwidth Wall

On September 27, 2026, developer JCode posted a tweet showing that a 32GB Mac mini M6 running Qwen3.8-27B (MLX 4-bit) with DFlash2 draft generation achieved 61.9 tokens per second in code scenarios — a 6.1× speedup over the 10.1 tok/s baseline — and 30.8 tok/s (3.0×) in chat. The benchmark used three random seeds (median), 64-token responses, temperature 1.0, top-p 0.95, top-k 20, 16K context, single stream. Critically, draft output matched serial decoding byte-for-byte (SHA-256 level).

Why the Bandwidth Wall Blocks Serial but Not Speculative Decoding

The Mac mini M6 has 170 GB/s unified memory bandwidth. A 27B model at 4-bit quantization occupies ~16 GB. Serial decoding must read all weights for every token: 170 ÷ 16 ≈ 10 tok/s theoretical ceiling, matching community measurements of 8.3–9.8 tok/s. Bandwidth is a physical constraint.

Speculative decoding changes the math: a small 2B draft model predicts 8 candidate tokens in one pass; the 27B target model verifies all 8 in a single forward pass. Verifying 8 tokens costs nearly the same bandwidth as verifying 1. If 5 are accepted, one weight read yields 6 tokens — the bandwidth cost shifts from “weights × steps” to “weights × verification rounds,” effectively multiplying throughput.

Code scenarios see 6.1× speedup because code is highly repetitive (variable names, indentation, templates), so the draft model predicts accurately. Chat and prose are less predictable, yielding only ~3×.

TensorFold: Lossless Speculative Decoding Engine

TensorFold (open source, Apache 2.0, by ashhart) runs on Apple Silicon and NVIDIA GPUs. Its key guarantee is lossless output: a draft token is accepted only if it exactly equals the token that serial sampling would produce. Users can verify by resending any request with "draft": false and comparing replies.

Hand-written Metal/CUDA kernels per model family, not a generic engine.

Supports Qwen3.8-27B, Qwen3.8 Flash Next, Nemotron 3.5 Lightning, GLM-5.3-Flash, Gemma 4 26B-A4B, DeepSeek-V4-Flash.

OpenAI-compatible API at http://127.0.0.1:8080/v1; coding agents only need to change the base URL.

Native binary from v1.0; install via brew install ashhart/tensorfold/tensorfold.

Version-sensitive: lock MLX to 0.31.2; newer MLX versions may fail the exactness check at load time.

DFlash2: Parallel Block Diffusion Draft Model

DFlash2 (Inco AI, August 2026) uses block diffusion : it predicts an entire block of tokens in parallel via one forward pass, then a tiny 2M-parameter selector picks a coherent path. Compared to DSpark (77.8M selector, +9.6% latency), DFlash2 adds ~1% latency and yields 20%+ more tokens per verification round, boosting benchmarks 16–25%.

The z-lab Qwen3.8-27B-DFlash2 draft model is only 2B parameters, 1.1 GB at 4-bit — negligible on a 32GB machine. DFlash2 is now integrated into SGLang, vLLM, llama.cpp (PR #27342), and oMLX, enabling draft generation across all Macs, not just high-memory flagships.

Qwen3.8-27B itself helps: 48 of its 64 layers are Gated DeltaNet linear attention, only 16 are full attention, so KV cache is tiny — enabling 16K context on 32GB.

Three-Step Replication

brew install ashhart/tensorfold/tensorfold
tensorfold pull TensorFold/Qwen3.8-27B-MLX-4bit
tensorfold pull z-lab/Qwen3.8-27B-DFlash2
tensorfold serve TensorFold/Qwen3.8-27B-MLX-4bit \
  --drafter z-lab/Qwen3.8-27B-DFlash2 \
  --context 16384 --temperature 1.0 --no-thinking

Point any OpenAI client to http://127.0.0.1:8080/v1. Add "draft": false to a request to verify lossless output.

Cross-Machine Benchmarks (Self-Reported / Community)

Higher bandwidth raises both serial baseline and speculative absolute throughput. M6’s 6.1× multiplier is the highest ratio, but absolute 61.9 tok/s still feels like “streaming keeps up with typing,” not the “screen-by-screen burst” of M5 Max.

Caveats

Self-reported data. 61.9 comes from project benchmarks and community tweets; methods and seeds are transparent but no independent third-party replication. Chat 3.0×, prose even lower (one 32GB MacBook saw only ~10% gain). Do not extrapolate 6.1× to all scenarios.

Code is the sweet spot. For Q&A or writing, expect 1.5–3×; for coding agents doing bulk edits, 6× is real.

Context length pressure. 16K was the test setting. Model uses ~16 GB, system takes more; longer contexts risk OOM. Native 262K is a model capability, not a guarantee on this hardware.

M6 is not the raw speed king. M4 Pro mini (273 GB/s) is 60% faster serially; M5 Pro mini (307 GB/s, 64 GB max) is 1.8× faster, but costs $400+ more. M6’s advantage is 32GB unified memory (fits 27B family) plus quiet, low-power always-on experience.

Conclusion

The tweet’s real significance is not “M6 suddenly got faster,” but that speculative decoding has reached the $6,999 entry-level Mac : open-source engine, 1.1 GB draft model, one-line brew install, lossless speedup with byte-level verifiability. The bandwidth wall remains at 170 GB/s, but how fast you climb it is now a software decision — the most compelling local-inference trend of 2026.

Data sources: JCode tweet & benchmark chart (2026-09-27, seeds 1234-1236 median, 64 tokens, T=1.0/top-p 0.95/top-k 20, 16K context, single stream); TensorFold GitHub (ashhart/TensorFold, Apache 2.0); Inco AI blog “DFlash 2: Keep Drafting Parallel” (2026-08); z-lab/Qwen3.8-27B-DFlash2-GGUF model card; RunAIHome “Apple M6 Mac Mini for Local AI in 2026”; Bryar World 32GB MacBook comparison; AI Hacker Daily 2026-09-28 archive. Cross-machine data are project/community self-reports as noted.