AI Agents Break the Web's 30-Year Traffic Contract: What Replaces It?

The Collapse of the Web's Implicit Give/Get Contract

Cloudflare co-founder and CEO Matthew Prince, drawing on the company's view of global internet traffic — covering over 40% of the top 10,000 sites and processing 500 million requests per second — argues that the rise of large language models has broken the unwritten "Give/Get" bargain that powered the web for nearly 30 years. Traditional search crawlers (Give) indexed pages in exchange for sending human visitors back to those sites (Get), enabling ad and commerce monetization. Answer engines now crawl massive numbers of pages but return almost no traffic.

Quantifying the Breakdown: Crawl-to-Visitor Ratios

Prince disclosed striking ratios: Google today crawls 40 pages per visitor sent; ChatGPT crawls 3,000 pages per visitor; Anthropic crawls 100,000 pages per visitor. This means publishers relying on Google face 20× harder monetization, and the gap widens further with model crawlers. Because agents do not click ads, the ad-driven model — already criticized for incentivizing clickbait and outrage — loses its foundation entirely.

Impact Beyond Commercial Publishers

The damage extends to non-commercial creators: hobbyists, Wikipedia volunteers, and other individuals who create for reputation, self-expression, or the satisfaction of being read. When their content is distilled into answers without any reader returning to the source, the psychological reward vanishes, reducing future creation incentives. Prince distinguishes institutional publishers (profit-driven) from the broader creator ecosystem (mixed motives).

Agent Commerce: From Research Assistant to Autonomous Buyer

Prince notes Agent Commerce is still early; today AI acts mainly as a research assistant that gathers and compares options, while humans make final purchase decisions. However, as agents advance to execute transactions, the dynamics shift. In a fully agent-mediated marketplace, ranking becomes absolutely rational — based on specs, price, reviews — eliminating the serendipity and brand-discovery moments that let small offline businesses and new brands compete against incumbents.

Toward a New Value Exchange: Content Independence and M2M Micropayments

The article raises the concept of a "Content Independence Day" — a new settlement layer where machine-to-machine (M2M) micropayments compensate creators per crawl or per answer use. This poses massive challenges for existing financial clearing systems, which are not built for billions of tiny, high-frequency transactions between automated agents.

Key insight: The problem is not the old institutions but the core values — news production, knowledge creation, narrative expression — that need new protection mechanisms.

Cloudflare's infrastructure position gives it a unique vantage point to observe these traffic shifts and advocate for technical solutions that restore a sustainable value loop between crawlers, content creators, and commerce.