Can Transformers Lead to AGI? Sergey Brin Says Yes—But They're Already Evolving

Sergey Brin, who rarely gives public remarks, sat down for an unscripted Q&A at Google DeepMind Build Day in early June, focusing on "frontier AI." He spoke about AI’s growing integration with scientific computing, the boundaries of superintelligence, the limits of Transformers, and the trade‑offs of Gemini on the programming track.

Brin emphasized that AI models from different specialties are converging. Historically Google trained separate models for each scientific problem, but today its flagship Gemini model can simultaneously achieve industry‑leading performance on mathematics and other scientific tasks, a convergence he did not anticipate.

He highlighted a "transfer" phenomenon in model training: improving one skill often lifts seemingly unrelated abilities. For example, code‑training boosts mathematical reasoning, while image‑training enhances performance on geometric word problems. These cross‑skill gains arise naturally during training rather than from deliberate engineering.

When asked about prompt design, Brin admitted uncertainty about whether to issue concrete debugging commands or higher‑level algorithmic requests, noting that even within Google the understanding of Gemini’s capability boundaries remains incomplete.

He also described the "chain‑of‑thought" prompting technique, which initially lacked solid theoretical backing but has repeatedly proven to markedly improve model performance, becoming a key driver of recent AI advances.

Regarding superintelligence, Brin rejected the idea that it could solve classic problems like P vs NP, stating that the prevailing academic consensus (P ≠ NP) means such problems remain unsolvable regardless of model strength. He defined superintelligence as intelligence surpassing human levels, not the ability to resolve mathematically undecidable tasks.

Brin offered two perspectives on AGI: one that views AGI as a self‑improving AI system, and a broader definition where AGI can perform any task a human can, which would require robust world‑modeling and robotics capabilities.

When asked how humanity should respond to accelerating model abilities, Brin mentioned brain‑computer‑interface research but said he prefers to wait until the technology matures before adopting it, noting that current model capabilities do not yet justify altering human physiology.

He cited historical examples—Deep Blue’s victory over Kasparov and AlphaGo’s win over top Go players—to argue that AI surpassing human performance in specific domains does not eliminate those fields; instead, it raises overall human expertise.

On the question of whether the Transformer architecture can support AGI, Brin leaned toward a positive answer, pointing out that Transformers have demonstrated adaptability far beyond their original text‑processing purpose, now being widely used for image and video tasks. He noted, however, that today’s Transformers have diverged significantly from the original paper’s design.

Internally, Google is investing heavily in "AI building AI," including model‑monitoring of training processes and automated data generation, work Brin describes as "self‑improvement" and which occupies most of his time.

Brin acknowledged that Google started late on code‑centric AI capabilities. While Gemini 3.0 and 3.1 were industry‑leading on many metrics six months ago, competitors have since made noticeable gains in complex and long‑running programming tasks. Google’s Flash series remains strong in low‑latency, interactive scenarios, but overall the company should have prioritized code‑ability research earlier.

Regarding his own role, Brin said product delivery is handled by Corey and Demis, while he focuses on continuously questioning the team, ensuring priorities are followed, and highlighting overlooked directions—sometimes disrupting the team’s rhythm.

He observed that monthly lab rankings are volatile and can appear pessimistic, but over longer periods labs alternate leadership across technical directions, which he sees as normal. Overall, Brin holds an optimistic view of Gemini’s current position.

Reference links: https://www.youtube.com/watch?v=gsv5o8ANdDo ; https://x.com/jaynitx/status/2072691938738987497?s=20