Inkling 2 Mini: 26B Model Runs on 32GB Mac, Matches 27B in Agentic Coding
Inkling 2 Mini, a 26.4B parameter MoE model with recurrent layers, runs on 32GB Macs and matches 27B models on agentic coding benchmarks but falls short in multi-turn chat and hard reasoning, with limitations in knowledge retention, safety, and non-English output.
Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, released Inkling 2 Mini on November 20 under Apache 2.0 on Hugging Face. The model has 26.4B total parameters and approximately 20GB weights, targeting local execution on 32GB unified-memory Macs.
Three Parameter Counts
The article clarifies three distinct parameter metrics that often cause confusion:
Total parameters: 26.4B — Memory footprint; determines the ~20GB weight size and 32GB memory requirement
Single activation: 3.4B — Parameters activated per token without recurrence
Per-token weight activation: 5.4B — Parameters actually activated per token when middle layers run three times; determines inference speed
Key rule: Total parameters govern memory; activation governs speed. The 26.4B total is the admission ticket for hardware.
Note: The earlier Inkling 2 Small (594B total / 17B active) is a completely different model.
Recurrent Architecture: Same Weights, Three Passes
24 layers total, grouped into 4 blocks of 6 layers each.
Middle 12 layers (layers 7–18) execute three times per token; first and last 6 layers run once.
Effective compute depth: 48 layers per token, but only 24 layers of weights stored.
Each layer uses Mixture of Experts (MoE): 128 experts, top 8 selected per token.
Analogy: Instead of writing an essay once, the model rewrites the core argument three times while using the same scratch paper. Benefit: deeper reasoning without extra memory. Cost: slower per-token latency.
Official statement: Recurrence helps only on reasoning tasks (nested retrieval, multi-hop QA). Pure knowledge benchmarks (AA-Omniscience, SimpleQA) do not improve because knowledge capacity remains bound by total parameters (26B). The model compensates with tool use, leading on BrowseComp, DeepSearchQA, and MCP Atlas.
Benchmarks: Mostly First, Two Clear Losses
All scores below are self-reported by Thinking Machines; independent replication is pending.
AA Intelligence Index v5.0: Inkling 2 Mini 20.1 vs Qwen3.8-27B 20.1 — Tie
DeepSWE v1.1: Inkling 2 Mini 48.2% vs Qwen3.8-27B 42.2% — Mini wins
SWE-bench Pro v2 (Hard): Inkling 2 Mini 77.9% vs Qwen3.8-27B 73.8% — Mini wins
SWE-bench Verified: Inkling 2 Mini 79.2% vs Qwen3.8-27B 77.8% — Mini wins
Terminal Bench v2.1: Inkling 2 Mini 71.6% vs Qwen3.8-27B 73.0% — Qwen wins
Multi-turn Dialogue (MultiChallenge): Inkling 2 Mini 50.0% vs Qwen3.8-27B higher — Qwen wins
Complex Instructions (ComplexConstraints): Inkling 2 Mini 33.4% vs Qwen3.8-27B 35.6% — Qwen wins
Hard Reasoning (CritPt v2 / HLE): Official admits lagging — Qwen wins
Summary: Inkling 2 Mini matches or beats 27B on agentic coding (SWE-bench, DeepSWE) but loses on multi-turn chat, complex instruction following, and hard reasoning.
Third-party data from Artificial Analysis: Mini averages $0.59 and 9 minutes per intelligence-index task via API; Qwen3.8-27B averages $3.53 and 38.2 minutes. This reflects API cost/latency, not local speed.
Mac Deployment: Three Official Demonstrations
32GB unified memory is the official local floor. Runs via Ollama (pre-release 0.40.17) or NVIDIA DGX Spark. Weights ~20GB (mostly NVFP4, last 4 layers MXFP8), leaving just enough headroom on 32GB.
Offline flight-mode demo: Researcher on a 48GB M5 Pro MacBook Pro built a playable flight simulator in minutes without internet.
20-agent simulation on one offline Mac: Thinkyville pixel town with 20 villagers, each planning, chatting, and running a baking contest, all driven by a single Mini instance.
Caveats: No official tokens-per-second numbers released; community claims of speedups over 4-bit quantization are unverified. Weights are already quantized (NVFP4/MXFP8), simplifying deployment — no GGUF hunting needed.
Four Limitations to Know Before Running
Not a chat model. Multi-turn dialogue and heavily constrained prompts are explicitly weaker than Qwen3.8-27B. Better alternatives exist for daily chat assistants.
Knowledge capacity unchanged. Recurrence adds thinking passes, not stored facts. Hard knowledge tasks will not surpass larger models.
Agent safety risks. Official acknowledgment: the model can enter self-check loops during debugging and may delete or overwrite files without confirmation when using tools. Do not point it at critical directories.
Non-English output degrades. Chinese output may be truncated or mixed with English — a significant concern for Chinese users.
Decision Guide by Hardware
32GB+ Mac: Officially supported tier. Try if you want to experiment with agentic coding and accept pre-release Ollama and the four limitations above.
16GB / 24GB Mac: Do not attempt; 20GB weights exceed memory. Wait for community quantized versions.
Just want the capability: Use API or hosted Tinker. Official pricing: $0.15/M input tokens, $0.50/M output tokens. Artificial Analysis measured average task cost at $0.59.
The core innovation is the recurrent architecture — reusing the same weights for multiple reasoning passes to lift a 26B model to near-27B agentic performance. It is not a universal small model, but it concretely lowers the barrier for running local agents to a single 32GB laptop.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
