Inference Set to Consume 70% of AI Compute Power, Leaving 30% for Training

Speaking at the 2026 China AIGC Industry Summit, Fusion Fund founding partner Zhang Lu highlighted a fundamental shift in the AI narrative: inference is overtaking training as the dominant source of compute demand. Historically, training consumed over 70% of AI compute, but recent trends show inference already representing 50% and expected to rise to a 70:30 split (inference:training) as intelligent agents replace simple chat interactions.

Because inference is a continuous, long‑term workload, its power consumption will become the primary driver of AI infrastructure. Zhang notes that the communication layer inside AI data centers can consume tens to hundreds of times more electricity than the actual computation, making next‑generation optical communication technologies far more critical than previously recognized.

The real bottleneck now lies in the data layer. While architectures and raw compute are mature, the scarcity of high‑quality, real‑world data hampers progress. Synthetic data can supplement but cannot replace edge‑case data collected from physical environments. Consequently, investment is flowing into new data‑collection platforms, data curation tools, and data libraries that can feed high‑fidelity datasets into AI models.

Physical AI—encompassing simulation, world models, and direct interaction with three‑dimensional sensor data—is emerging as a key frontier. Innovations such as flexible electronics and artificial skin sensors (e.g., Stanford‑developed high‑precision, low‑power tactile gloves) aim to capture rich physical data while minimizing energy costs associated with moving that data.

Edge computing is another pillar of the new AI stack. Small models that fit on devices like a Raspberry Pi (fewer than 1 billion tokens) can deliver GPT‑4‑level capabilities locally, reducing latency and data‑transfer overhead. This approach is especially valuable for regulated sectors where privacy and data sovereignty are paramount.

Application‑level opportunities are concentrated in three domains: medical AI, space AI, and nano‑/microrobotics. The medical sector offers dense, high‑quality data and is attracting multi‑billion‑dollar collaborations (e.g., a $1 b partnership between Lilly and Nvidia). Space AI will underpin future space factories and refueling stations, while microrobots and nanorobots promise breakthroughs in targeted drug delivery and vascular interventions.

Industry investment is accelerating dramatically: Fortune‑500 AI budgets have jumped from tens of millions to billions of dollars, and procurement cycles have compressed from six months to one or two months. This rapid integration speed, combined with the need for high‑quality data and optimized inference infrastructure, defines the competitive edge for AI companies in the coming years.