Skip to content
“10 of 13” Is Fake: Inkling 2 Mini vs Qwen3.8-27B Benchmark Fact-Check & Mac Guide

“10 of 13” Is Fake: Inkling 2 Mini vs Qwen3.8-27B Benchmark Fact-Check & Mac Guide

The author fact-checks a viral benchmark chart claiming Inkling 2 Mini leads Qwen3.8-27B on 10 of 13 tasks, finds the actual count is 9–4, identifies two data discrepancies with the official model card, explains architectural reasons for each model's strengths, and advises Mac users on which model to run based on memory and use case.

Lao Guo's Learning Space
Lao Guo's Learning Space
Lao Guo's Learning Space

The article investigates a widely shared benchmark image comparing Inkling 2 Mini (26.4B MoE, 5.4B active parameters per token) against Qwen3.8-27B (27B dense). The image bears a badge stating “Ahead on 10 of 13,” but the author counts each of the 13 benchmarks individually and finds Inkling wins 9, Qwen wins 4 — a 9:4 split, not 10:3.

Source of the Image

The image carries a watermark from X user @MiaA_L_Jab and a footnote:

Source: Thinking Machines, "Introducing Inkling 2 Mini" (Nov 20, 2026). Vendor-reported; eval setups vary.

This means the numbers are self-reported by the vendor , not independently reproduced. Artificial Analysis’s independent intelligence index scores both models at 20.1, indicating parity.

Badge Count Discrepancy

The author provides a table (Image:

Table A: 13 benchmark item-by-item comparison
Table A: 13 benchmark item-by-item comparison

) showing the 13 benchmarks. Counting row by row yields 9 wins for Inkling, 4 for Qwen . The badge’s “10 of 13” overstates by one; the marginal item is likely FrontierCode v2 (15.1 vs 14.9) , a 0.2‑point difference.

Two Numeric Mismatches with the Official Model Card

The author cross‑references the image values with Thinking Machines’ official model card and finds two inconsistencies (Image:

Table B: Three suspicious points
Table B: Three suspicious points

):

DeepSearchQA Qwen value: Image shows 70.7; official card lists 79.7 . This inflates Inkling’s lead from 5.6 to 14.6 points.

Last row label: Image reads “EnterpriseOps-Gym (42.0 vs 44.2)”; official card shows GDPVal-AA (Elo 1319 vs 1427) — different benchmark, different units (Elo vs percentage).

Most other numbers (DeepSWE, SWE‑bench series, BrowseComp, MCP Atlas, Terminal‑Bench) match the official card.

Strength Breakdown After Removing Inflation

Inkling 2 Mini excels at agentic programming and tool use: DeepSWE (48.2 vs 42.2), SWE‑bench series, BrowseComp (79.0 vs 61.6), MCP Atlas (82.7 vs 77.3), DeepSearchQA. The official card attributes this to its recurrent “loop” architecture — the middle 12 layers repeat computation 3 times per token, allowing deeper reasoning for agent tasks.

Qwen3.8-27B excels at knowledge work and hallucination resistance: GDPVal-AA (knowledge deliverables), AutomationBench, AA‑Omniscience (hallucination rate 71.4% vs 63.4%), MultiChallenge (50.0%), ComplexConstraints (33.4%). The official explanation: recurrence adds “thinking passes” but cannot increase stored knowledge, which is bounded by total parameters (26B).

In short: Inkling is the “hands‑on” model; Qwen is the “reliable knowledge” model.

Which Model Should Your Mac Run?

Decision table (Image:

Table C: Which model for your Mac
Table C: Which model for your Mac

):

Agentic coding / agents / tool calling: Inkling 2 Mini. Requires ≥32GB RAM (weights ~20GB). Runs via Ollama (pre‑release) or MLX.

Daily chat / knowledge QA / Chinese output / low hallucination: Qwen3.8-27B. Official card notes Inkling may truncate or mix Chinese/English — a hard constraint for Chinese users.

16/24GB RAM: Neither fits; Inkling weights alone are ~20GB, Qwen 27B dense needs even more.

Free‑tier via API: Inkling API priced at $0.15/$0.50 per million tokens; Artificial Analysis measured $0.59 per query, 9 minutes vs Qwen’s $3.53, 38.2 minutes. However, Inkling generates ~38% more output tokens, increasing local inference latency.

Three Rules for Reading Any Benchmark Chart

Check attribution: Vendor‑reported vs independent third‑party — the footnote “Vendor‑reported” is a disclaimer.

Count the badge: “Leads N of M” summaries are often wrong; count each row yourself.

Verify against the source: Key numbers should be cross‑checked with the official model card — even official cards sometimes misstate parameter counts (e.g., 26.4B written as “26 billion”).

The article concludes that despite the inflated badge and two data errors, the core conclusion holds: Inkling 2 Mini’s recurrent architecture genuinely brings agentic coding performance into 27B dense territory at 1/5 the active parameters, making it a viable local agent model for 32GB+ Macs.

Disclaimer: Benchmark image sourced from X account @MiaA_L_Jab, data cited from Thinking Machines’ “Introducing Inkling 2 Mini” (2026‑11‑20), vendor‑reported, not independently replicated. The 13 benchmark values were OCR‑extracted and color‑classified by the author; badge “10 of 13” manually counted as 9:4; DeepSearchQA Qwen value (70.7) vs official card (79.7) mismatch; GDPVal‑AA (Elo 1319/1427) vs image’s EnterpriseOps‑Gym unit mismatch. Inkling 2 Mini specs (26.4B MoE / 3.4B active / 5.4B per‑token activation / 24‑layer loop / 1M context / Apache 2.0 / ~20GB weights / 32GB official minimum) from official model card and Hugging Face page. Artificial Analysis intelligence index 20.1 (tie) and cost/latency from independent evaluation. This is technical analysis, not purchase advice.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

model evaluationMaclocal LLM deploymentagentic programmingfact-checkingLLM benchmarksQwen3.8-27BInkling 2 Mini
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.