Token Efficiency: The Critical Lever for Sustainable AI Computing

The report Token Efficiency: Key Variable for Green Computing in the AI Era , jointly released by Ant Group and the China Academy of Information and Communications Technology (CAICT) AI Research Institute, positions Token efficiency as the central metric for sustainable AI infrastructure.

Three Structural Challenges in AI-Era Green Computing

The report identifies three interlocking challenges that extend the traditional green-computing chain from "carbon → electricity → compute → business" to "carbon → electricity → compute → Token → business":

Challenge 1 – GPU Usability Gap: Domestic intelligent computing centers average GPU utilization below 30%, with some enterprise centers under 10%. Nominal compute expansion has not translated into effective compute release.

Challenge 2 – Token Consumption Explosion: China's enterprise Token consumption is projected to grow from ~114 trillion in 2024 to ~40,000 trillion in 2026, a 300× increase in two years. Data centers are irreversibly shifting from "compute centers" to "Token centers."

Challenge 3 – Energy & Carbon Pressure: Global data-center electricity use is forecast to double from ~415 TWh (2024) to ~945 TWh (2030), with AI as the primary driver. 2025 data-center carbon emissions reached 286 million tons, 57% above prior estimates.

Token efficiency — the intelligence output per Token — sits at the hub connecting upstream compute/energy investment to downstream business value.

Four-Dimensional Definition of Token Efficiency

The report defines Token efficiency across four dimensions, emphasizing optimization under effectiveness constraints rather than mere Token reduction:

Intelligence Output: Valid intelligent output per Token consumed.

Economic: Business value generated per Token cost.

Experience: Speed of Token processing and generation.

Energy/Carbon: Effective intelligent output per unit of energy and carbon emitted.

Four-Layer Full-Stack Technical Framework

Following the Token lifecycle from production to consumption, the report constructs a four-layer framework where single-layer optimization hits a ceiling; only full-stack coordination approaches end-to-end optimality.

Model Layer – More Intelligence per Token: Sparse attention, MoE architectures, on-demand reasoning, quantization and distillation to raise intelligence density at the source.

Inference Layer – Less Compute per Token: KV Cache and Prefix Cache optimization, continuous batching with Token-budget-aware scheduling, prefill-decode (PD) separation, speculative decoding, inference-engine acceleration.

Agent Layer – Control Task-Execution Token Waste: Planning and execution optimization, tool-call Token optimization, context and memory management, multi-session global optimization.

Application Layer – Spend Tokens Where Business Value Is Created: Prompt and context optimization, semantic caching and result reuse, model routing and invocation strategy, Token consumption governance with feedback loops.

Two Typical Practices Reveal Differentiated Governance Patterns

The report examines two contrasting scenarios to demonstrate that no universal template exists; optimization must respect industry constraints and business stage.

SME AI Applications – "Reasonable Token Spend" via Four Application-Layer Defenses: Core problem is uncontrolled spend from full-flagship-model usage and high-frequency repeated calls. The emerging practice deploys four defenses under a general Agent architecture — model tiering with intelligent routing, caching and skill solidification, context and retrieval governance, budget circuit-breakers and observability — without touching chips or models. Result: Token spend compressed to 20-40% of original.

Medical Rigorous Scenario (Ant Group's "Afu") – Prudent Cost Reduction Under Strong Constraints: Under privacy, stability, and UX constraints, long-dialogue historical evidence-chain compression achieved a 4.5× Token efficiency improvement while maintaining effectiveness, reaching leading scores on LongBench v2 dialogue and Agent tasks.

Both cases converge on the principle: Token efficiency optimization requires differentiated governance aligned with industry constraints and business phase.

Five-Dimensional Measurement System for Quantifiable Green Value

"If you cannot measure, you cannot improve." The report proposes a measurement system whose logic progresses "How much spent → Is it worth it → Is it green":

Compute Efficiency: Compute consumed per Token.

Context & Token Utilization Efficiency: Effectiveness of context and Token usage.

Task Effectiveness: Task quality and Token parsimony.

Economic Efficiency: Cost vs. business value.

Energy/Carbon Efficiency: Energy and environmental cost per effective intelligent output.

The five dimensions form a complete transmission chain from compute input to carbon outcome, enabling a calculable operational-stage carbon-abatement model that makes Token efficiency's green value quantifiable, disclosable, and traceable .

Industry Paradigm Shift and Co-Governance Recommendations

The report notes AI competition is shifting from "scale race" to "efficiency competition," with focus moving from model scale to Token efficiency and further to per-unit-intelligence energy and carbon footprint. It calls for multi-stakeholder co-construction of a Token efficiency and green-computing governance system so that every Token processed is more efficient, every compute investment creates greater value, and every intelligence leap avoids excessive energy and carbon cost.