Token Efficiency: Key Variable for Green Computing in the AI Era

Introduction: Token Efficiency as the Critical Metric for Green AI Computing

Ant Group and the China Academy of Information and Communications Technology (CAICT) Artificial Intelligence Institute jointly released the research report "Token Efficiency: Key Variable for Green Computing in the AI Era." The report argues that large models are reshaping the computing industry, driving a sharp rise in compute and energy consumption. As the unit of measurement shifts from FLOPS to tokens, making each token carry higher intelligent value at lower energy-carbon cost becomes the central challenge for green computing in the AI era.

Three Structural Challenges Facing Green Computing in the AI Era

The report extends the green computing philosophy—"release more effective compute, let each unit of compute do more work with less emissions"—and identifies three new structural challenges:

Challenge 1: GPU usability conversion bottleneck. Despite rising nominal GPU compute, actual effective compute release faces usability conversion difficulties. Domestic intelligent computing centers average GPU utilization below 30%, with some enterprise centers under 10%. Rapid nominal scale expansion has not translated into synchronous effective compute release.

Challenge 2: Explosive token consumption driving compute/storage demand. China's enterprise token consumption is projected to grow from ~114 trillion in 2024 to ~40,000 trillion by 2026, a 300x increase in two years. Data centers are irreversibly shifting from "compute centers" to "token centers."

Challenge 3: Intelligent compute energy far exceeds general compute, squeezing "less emissions" from both supply and demand. Global data center electricity consumption is expected to double from ~415 TWh in 2024 to ~945 TWh in 2030, with AI as the primary growth driver. Large models have become the fastest-growing carbon emission sector in the digital industry—2025 global data center carbon emissions reached 286 million tons, 57% higher than previous forecasts.

These three challenges point to a deep contradiction: continuously swelling intelligent demand versus structural imbalance between compute efficiency and energy supply capacity. The report proposes that the green computing conversion chain has expanded from "carbon→power→compute→business" to "carbon→power→compute→token→business." The token stage sits between compute and business, acting as the core hub connecting compute input to business output. Its conversion efficiency directly determines whether upstream compute and energy investments effectively translate into downstream business value. Consequently, the efficiency perspective must expand from "compute utilization efficiency" to "intelligent output efficiency," making token efficiency the key variable for green computing in the AI era.

Four-Dimensional Definition of Token Efficiency

The report defines token efficiency as a multi-dimensional concept covering:

Intelligence Output – efficiency of converting token consumption into effective intelligent output.

Economic – cost efficiency of converting token consumption into business value.

Experience – speed efficiency of token processing and generation.

Energy-Carbon – energy and carbon efficiency of converting energy input into effective intelligent output.

Improving token efficiency is not simply about "saving tokens" but seeking optimal efficiency under effectiveness constraints, ultimately supporting affordable, high-quality, and sustainable intelligent services.

Four-Layer Full-Stack Technical Framework for Token Efficiency

Following the token flow from production to consumption, the report constructs a full-stack technical framework across four layers:

Model Layer – Let each token carry more intelligence. Technologies: sparse attention, MoE architecture, on-demand thinking, quantization and distillation. Goal: increase token intelligence density at the source, reducing tokens per answer.

Inference Layer – Let each token consume less compute. Technologies: KV Cache and Prefix Cache optimization, continuous batching with token-budget-aware scheduling, PD (prefill-decode) separation deployment, speculative decoding, inference engine acceleration. Goal: lower per-token energy and cost, increase generation speed.

Agent Layer – Control token flow loss during task execution. Technologies: planning and execution optimization, tool call token optimization, context and memory management, multi-session global optimization. Goal: ensure each loop consumes only necessary tokens.

Application Layer – Spend tokens where they generate real business value. Technologies: prompt and context optimization, semantic caching and result reuse, model routing and invocation strategies, token consumption governance and feedback loops. Goal: put tokens on the cutting edge.

The four layers mutually support each other; single-layer optimization hits a ceiling. Only full-stack collaboration can approach end-to-end token efficiency optimum.

Two Typical Practices Validate Differentiated Governance

The report examines two typical practices revealing differentiated token efficiency optimization paths across scenarios.

SME AI Applications: Four Defense Lines for "Reasonable Token Spend"

Core problem: uncontrolled token expenditure from "full-process flagship models" and high-frequency repeated calls. Industry practice has formed a general approach targeting "reasonable token spend" under a generic Agent architecture with four defense lines:

Model tiering and intelligent routing

Caching and skill solidification

Context and retrieval governance

Budget circuit breaker and observability

Without touching chips or modifying models, these four application-layer defenses alone compress token spend to 20-40% of original.

Medical Rigorous Scenario: Prudent Cost Reduction Under Strong Constraints

Using Ant Group's "Afu" as an example, under multiple hard constraints—privacy security, effect stability, user experience—the system achieves prudent cost reduction. Through long-conversation history evidence chain compression technology, token efficiency improves 4.5x while maintaining performance, reaching leading levels on LongBench v2 dialogue and Agent tasks.

Both practices point to a fundamental judgment: Token efficiency optimization has no universally portable template; it must be differentiated based on industry constraints and business stages.

Five-Dimensional Measurement System Driving Multi-Stakeholder Governance

"If you cannot measure, you cannot improve." The report proposes a token efficiency measurement system from a green computing perspective, covering five dimensions:

Compute Efficiency

Context and Token Utilization Efficiency

Task Effectiveness

Economic Efficiency

Energy-Carbon Efficiency

The internal logic progresses in layers: "How much spent → Worth it → Green." Compute efficiency answers "how much compute per token," task effectiveness answers "how well and how efficiently the task is done," economic efficiency answers "cost vs. business value," and energy-carbon efficiency ultimately answers "energy and environmental cost." Together they form a complete transmission chain from compute input to carbon emission results, further enabling a calculable operational carbon reduction model that makes token efficiency's green value quantifiable, disclosable, and traceable.

Conclusion and Outlook

The report notes that AI industry competition is undergoing a paradigm shift from "scale race" to "efficiency competition," with focus moving from model scale to token efficiency and gradually extending to energy and carbon emissions per unit of intelligent output. It recommends multi-stakeholder collaboration to build a governance system for token efficiency and green computing, ensuring every token processing is more efficient, every compute investment creates greater value, and every intelligent leap no longer comes at the cost of excessive energy consumption and carbon emissions.