Why Leading AI Labs Are Racing to Build Their Own Inference Chips

On July 7, two separate reports highlighted that Chinese AI firms DeepSeek and Zhipu were evaluating custom chips, prompting a broader look at why leading AI labs worldwide are pursuing in‑house silicon.

DeepSeek and Zhipu’s Early‑Stage Plans

Reuters cited three insiders that DeepSeek is quietly recruiting chip designers for an inference‑focused ASIC, while The Information reported Zhipu’s interest is driven by a 27‑fold token‑consumption surge in its GLM‑5.2 model on Vercel, with a projected two‑year development timeline.

OpenAI’s First ASIC – Jalapeño

OpenAI announced on June 24, together with Broadcom, the Jalapeño ASIC designed from scratch for large‑model inference. Hardware lead Richard Ho said the chip optimizes core, memory, networking and service patterns. Early lab tests claim per‑watt performance far exceeds the current state‑of‑the‑art, though third‑party benchmarks are pending.

Tom’s Hardware noted the design‑to‑tape‑out cycle took only nine months, accelerated by using AI to design the AI chip itself. Deployment is slated for late 2026 with a 10 GW capacity partnership with Broadcom, and Microsoft is expected to purchase about 40 % of the initial capacity.

Anthropic’s Cautious Move

Initially, Anthropic said it might only buy chips, but by early July it was in talks with Samsung for a 2 nm custom ASIC, hiring former OpenAI chip engineer Clive Chan. The company still emphasizes a diversified hardware stack (Nvidia, AWS, Google) as core to its compute strategy.

Why Inference, Not Training?

Industry analysis (Introl) shows inference now consumes roughly two‑thirds of AI compute, shifting the cost focus from one‑off training to continuous serving. ASICs excel at this fixed, high‑throughput workload, whereas GPUs remain the flexible tool for training.

Agent‑driven applications amplify inference demand: each user goal can trigger dozens to hundreds of inference calls, turning inference into a “loop cost” rather than a simple “query cost.”

Economic Drivers

Midjourney’s migration from Nvidia GPUs to Google’s TPU‑v7 cut monthly compute spend from $2.1 M to $0.7 M (≈65 % reduction). Scaling this saving to billion‑query operators makes multi‑billion‑dollar custom‑chip investments financially compelling.

TrendForce predicts custom ASIC shipments will grow 44.6 % in 2026 versus 16.1 % for GPUs, the first time ASIC growth outpaces GPUs. Bloomberg‑cited Introl forecasts the AI accelerator market to reach $604 B by 2033, with custom silicon’s share accelerating.

Strategic Control and Vendor Lock‑in

Major hyperscalers (Google TPU, Amazon Trainium, Microsoft Maia, Meta MTIA) already run internal inference loads on custom silicon, gaining cost, performance and supply‑chain control. Oplexa notes that owning the silicon roadmap, cost structure and supply chain creates a competitive moat that cannot be bought.

Switching away from Nvidia’s CUDA ecosystem incurs significant migration effort (2–6 weeks for vLLM to Neuron SDK) and may limit model support, reinforcing the incentive to develop proprietary ASICs.

Additional Constraints for Chinese Labs

Beyond the universal drivers, DeepSeek and Zhipu face U.S. export controls, adding regulatory risk absent for their Western peers. DeepSeek’s chip rumor coincides with a $50 B financing round, and Zhipu’s recent HK listing provides capital for the costly venture.

Risks and Outlook

Custom chip projects can require 18–24 months of design and massive upfront engineering effort, making them viable only for firms with stable, predictable workloads. For smaller players, Nvidia GPUs remain more cost‑effective.

TechTimes estimates a 40‑65 % cost advantage for high‑volume inference, but this benefit diminishes for low‑frequency users. Analysts expect ASIC shipments to possibly surpass GPU volumes by 2027, yet both markets will grow, with GPUs retaining dominance in training and diverse workloads.

Ultimately, the convergence of inference‑centric compute, exploding agent workloads, and desire to escape Nvidia’s pricing and software lock‑in drives AI companies toward custom inference ASICs, while export controls and high development costs pose significant hurdles.