Inkling – A 975‑Billion‑Parameter Open‑Weight Multimodal Model from Thinking Machines

Thinking Machines Lab, founded by former OpenAI CTO Mira Murati, announced the release of Inkling, a self‑developed AI model with a total parameter count of 975 billion and 41 billion active parameters. Inkling supports up to 1 million tokens of context and is fully open‑weight, allowing developers to download and fine‑tune the model.

Inkling was pretrained on 45 trillion tokens spanning text, images, audio, and video. It adopts a hybrid‑expert Transformer architecture: each Mixture‑of‑Experts (MoE) layer contains 256 routing experts and 2 shared experts, with six routing experts activated per token. Routing uses a sigmoid mechanism with a load‑balancing bias, and scores from routing and shared experts are jointly normalized for weighted output combination.

In the attention subsystem, Inkling alternates sliding‑window attention and global attention in a 5∶1 ratio, employing eight KV heads. Relative position embeddings replace RoPE, yielding better extrapolation to longer sequences. Two short‑convolution modules are inserted—one after the Key/Value projections of each attention layer and another before the MLP residual branch—to enhance local feature extraction.

The training pipeline combines a Muon optimizer for large‑matrix weights with Adam for remaining parameters, and ties weight decay strength to the square of the learning rate. After the initial pretraining phase, the model undergoes post‑training covering mathematics, agent programming, audio, vision, dialogue, and safety. A synthetic‑data supervised fine‑tuning stage, seeded by open‑weight models such as Kimi K2.5, precedes large‑scale asynchronous reinforcement learning, which performed over 30 million rollouts and maintained stable performance across two long‑duration training runs.

Benchmarking was conducted with inference intensity set to 0.99 and temperature 1.0. Inkling was evaluated on a suite of public tests—including Terminal Bench 2.1 (agent programming), HLE (high‑level reasoning), IFBench (instruction following), VoiceBench, MMAU, AudioMC (audio tasks), and FORTRESS (safety). On Terminal Bench, Inkling achieved comparable performance to Nemotron 3 Ultra while using only one‑third of the tokens, highlighting its cost‑efficiency. Across multimodal tests, Inkling demonstrated strong audio transcription, voice‑command execution, visual description, and visual‑reasoning capabilities, often ranking among the top open‑weight models.

Inkling also excels in practical generation tasks. It produced a one‑shot, fully functional web application with an embedded AI assistant capable of browser‑level interactions. In the Design Arena web‑development leaderboard, anonymous human judges placed Inkling among the strongest open‑weight models. The model generated multi‑page outputs with consistent visual style and successfully iterated a multiplayer snake game over 40 rounds based on GPT Codex feedback.

Inkling‑Small, a preview model with 276 billion total parameters and 12 billion active parameters, offers a lower‑latency alternative. Despite its reduced size, Inkling‑Small matches or exceeds Inkling on several benchmarks, thanks to a refined pretraining dataset and training scheme. Both models share the same scalable post‑training stack.

For customization, Inkling’s weights are accessible via the Tinker platform, which now supports fine‑tuning and provides an Inkling Playground for interactive exploration. The model can be invoked through APIs from Together AI, Fireworks, Modal, Databricks, and Baseten, and integrates with open‑source inference stacks such as SGLang, Miles, vLLM, TokenSpeed, llama.cpp, and Hugging Face Transformers. Full model checkpoints, including an NVFP4 checkpoint optimized for NVIDIA Blackwell systems, are hosted on Hugging Face.