What Ten Lessons Google Learned from a Decade of TPU Evolution?

Boosting Machine‑Learning Energy Efficiency and Reducing Carbon Footprint

Google’s measurements show that moving from early GPUs (e.g., P100) to modern TPUs reduces energy per operation by up to 100× and carbon emissions by up to 700× when combined with more efficient models, high‑efficiency data‑center PUE, and low‑carbon‑intensity locations.

Key factors

Model efficiency : The Primer model (released 2021) achieves the same quality as the original Transformer while using 4× less energy.

Hardware advances : TPU performance per watt is ~14× higher than the 2017 P100 GPU; each new TPU generation roughly doubles matrix‑multiply units while keeping die area modest.

Data‑center PUE : Google’s PUE is ~1.1, about 1.4× better than the industry average, further cutting operational energy.

Geographic energy mix : Training in locations with high renewable penetration (e.g., Oklahoma) can reduce carbon emissions an additional 9×.

Combined, these factors can lower ML energy consumption by ~80× and carbon emissions by ~700×.

Example: The GlaM mixture‑of‑experts (MoE) model uses a sparse activation pattern, invoking only ~8 % of its 1.2 trillion parameters per token. Compared with dense GPT‑3, GlaM’s accelerator runtime and energy are reduced by ~3×, and because training occurs in a clean‑energy data‑center, its total carbon footprint is ~14× lower.