Beyond CUDA: Nvidia’s Token Factory and Supply Chain Guard Its Moat from TPU

Beyond CUDA: Nvidia’s Token Factory

Nvidia CEO Jensen Huang explains that the company’s moat is not limited to its CUDA software stack. He describes a "token factory" where the input is electrons and the output is high‑value tokens, a process that requires deep technical creation, engineering capability, scientific knowledge, and design expertise, making the business hard to commodify.

Huang outlines Nvidia’s operating principle: "Do only what is necessary for the conversion, and do as little as possible otherwise." This leads to strict technical and engineering barriers for the parts Nvidia chooses to own.

Using a "five‑layer cake" analogy of the AI industry, Huang shows that Nvidia’s ecosystem spans all five layers, with core stages kept in‑house and non‑core stages handled through ecosystem partners, creating a vertically integrated supply chain that includes upstream fabs, memory and packaging providers, downstream PC manufacturers, application developers, and model providers.

Regarding the market belief that AI will commodify software, Huang argues the opposite: most software firms are tool makers, and the scale of tool usage is limited by the number of human engineers. As AI agents become more capable, the number of entities that can use software tools will rise, dramatically increasing tool‑call instances.

Huang predicts that the scaling of software tools and AI agents will drive massive growth in compute demand. Nvidia, as the carrier that converts electrons into tokens, will benefit from this sustained downstream demand.

When asked whether Nvidia’s core moat lies in locking future scarce chip component supply, Huang confirms that supply‑chain positioning is a key advantage but not merely a capacity lock. Public filings reveal Nvidia has signed procurement commitments worth close to $100 billion with fabs, memory, and packaging partners, and analysts estimate the total could reach $250 billion.

These commitments are based on Nvidia’s forecast of AI industry scale, allowing it to align with supplier decision‑makers and encourage them to invest heavily in capacity expansion. Suppliers cooperate because Nvidia’s large, predictable downstream demand can absorb the new production.

Huang also notes that events such as GTC speeches serve to communicate AI industry trends across the entire supply chain, aligning upstream and downstream participants for long‑term collaboration.

TPU Threat to Nvidia’s Competitive Position

Recent shifts in the global AI compute market show that the top three large models, Claude and Gemini, are trained on Google’s TPU, indicating TPU is emerging as a viable alternative to Nvidia GPUs.

The latest TPU‑8 series improves performance‑per‑dollar by 2.7×, prompting major AI players like Anthropic and OpenAI to allocate multi‑gigawatt capacity to non‑Nvidia platforms.

Huang responds that TPU is a fixed‑function ASIC optimized for tensor calculations, whereas Nvidia builds a programmable "accelerated computing" platform that can adapt to various frameworks and algorithms. This programmability offers resilience to changes in AI architectures, while ASICs lack such flexibility.