Inkling 2 Mini on MLX: 27 tok/s, 30GB Peak, but 26.4B Params Misreported 10x
Third-party benchmarks show Inkling 2 Mini running on MLX at 27 tok/s with 30GB peak memory on a 128GB Mac, but the official Chinese announcement misstates its 26.4B parameters as 2.6B, and the performance figures are conditional, not guaranteed for all hardware.
Yesterday I wrote about Inkling 2 Mini — Thinking Machines' 26B "looping" small model — with an official local requirement of 32GB memory. Today it actually runs: developers have produced an MLX-VLM adaptation , and a performance chart circulating in the community shows impressive numbers — 27.18 tok/s, 0.44s first-token latency, 30.19GB peak memory .
But the chart hides two pitfalls that can mislead you. The bottom line:
This performance comes from a third party (Neywa Labs) on their own hardware, not an official benchmark; and the official Chinese post writes "26.4B total parameters" as "26亿参数" (2.6 billion), confusing billion (B) with 亿 — the real total is ~26.4 billion (264亿), a 10× difference.
1. Correction first: 26.4B parameters, not 2.6B
The Chinese post from the official Thinking Machines account (@thinkmachines) states:
"Inkling 2 Mini is a looping transformer with only 2.6 billion parameters…"
This is a mistranslation . The original English reads:
"Inkling 2 Mini is a Mixture-of-Experts model with 26.4B total parameters , 3.4B active, and 5.4B weight activations per token to account for looping."
Here B = billion = 十亿 , so 26.4B = 26.4 billion (≈264亿) . The Chinese post reads "26.4B" as "26亿", treating billion as million and dropping a digit. The same chart's model info box correctly says "Parameters 26.4B total / 3.4B active".
Three parameter scopes, side by side:
Total parameters : 26.4B (≈264亿) — determines memory ticket
Single activation : 3.4B (≈34亿) — weights used per forward pass without looping
Per-token weight activations : 5.4B (≈54亿) — includes three loops, determines speed
Two numbers in one chart conflict; trust the English original and the model card, not the Chinese post.
2. What MLX-VLM actually did
MLX-VLM is an open-source tool (maintained by Neywa Labs, the team that posted the chart) for running multimodal models on Apple Silicon . Its Inkling 2 Mini adaptation highlights:
Loads native weights directly, no conversion : The official checkpoint uses mixed NVFP4 + MXFP8 precision; MLX-VLM reads it as-is, just place hf_quant_config.json alongside the weights.
Mini's weights are ~20GB (including the bundled DSpark draft model). This is the native mixed-precision size, not a user-quantized version.
Supports images : Pass --image <path> for vision-language input, also works text-only; tool calling via mlx_vlm.server.
DSpark speculative decoding : Add --draft-model thinkingmachines/Inkling-2-Mini to enable a small model to guess next tokens, verified by the large model for speedup.
One command runs it (using the official model ID):
mlx_vlm.generate --model thinkingmachines/Inkling-2-Mini --prompt "你好,介绍一下你自己" --max-tokens 256 --temperature 0Pure text needs no image; add --image <path> for vision; add --draft-model thinkingmachines/Inkling-2-Mini for DSpark acceleration.
3. Reading the performance chart: 27 tok/s is conditional
Breaking down each metric with its conditions:
Prefill speed : 1,439 tok/s — 8K prompt; chart notes ~2× faster than stock MLX
Decode speed : 27.18 tok/s — 8,192 tokens, DSpark enabled
DSpark speedup : +50.8% — relative to no speculative decoding; at 8K length
TTFT (first token) : 0.44s — 512-token short input
Peak memory : 30.19 GB — at 8,192 tokens
Three boundaries you must remember:
This is Neywa Labs' own test, not Thinking Machines' official benchmark. The chart labels "M5 Max · 128GB unified · Apple M10 GPU · Fully local" — run on a 128GB Mac. Your machine with less memory or shorter context will yield different numbers.
+50.8% is "at a specific length". Other lengths: 512 tokens +13.8%, 1024 +21.5%, 2048 +29.6%. Official MLX docs also state "DSpark speedup depends on workload"; don't treat peak as typical.
Decode speed varies with context length. The chart shows a bar series from 21.06 to 27.18 tok/s, not a constant; longer context increases memory and speed pressure.
4. The official docs' "disclaimer" matters more than the chart
I read the MLX-VLM Inkling2-specific documentation; several candid statements outweigh the pretty chart:
"Inference uses float activations, not the activation quantization of the reference implementation; no strict numerical parity with the reference implementation has been established." — MLX output may not match the official NVFP4 reference token-for-token.
"Prefill chunk size and speculative verification shape can change greedy decoding output." — Changing parameters can alter generated results and benchmark numbers.
"DSpark speedup depends on workload." — No fixed multiplier is promised.
Audio input and pipeline parallelism are not supported.
In short: 27 tok/s is "on this machine, this config, this length" — not a guarantee your Mac will hit that speed. This holds for any third-party local model benchmark.
5. Three judgments for you
32GB+ memory : Official supported floor. Weights ~20GB, measured peak 30GB; 32GB is "just enough", long contexts will be tight; 64GB+ is comfortable. Try the command above, start with 512-token short tasks to verify the pipeline.
16GB / 24GB : Don't force it. 20GB weights won't fit; no smaller quantized version released officially.
Want performance numbers : Treat the chart as an "upper-bound reference", not a promise. For real comparison, run on your own hardware with your own prompts.
Fair closing: Inkling 2 Mini landing on MLX matters not because of "27 tok/s", but because a 26B-parameter, vision-capable, 1M-context model can now run on a Mac with a single command for the first time . As for the Chinese post that turned 26.4B into "26亿" — even the official account can mistranslate billion; always re-read the original source before trusting any performance figure.
Data sources: Thinking Machines official release page & model card, Neywa-Labs/mlx-vlm repo (Inkling2 docs & performance chart, third-party test), Hugging Face model page. Performance scopes annotated inline.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
