Tang Jie Unveils GLM's AGI Roadmap: Pre-training, Post-training & Agent Scaling

Tang Jie's NUS Talk: GLM's Roadmap to AGI

Zhipu AI founder and Tsinghua professor Tang Jie recently presented at the National University of Singapore, outlining how the GLM series is progressing toward artificial general intelligence. The talk reviews the past five years of large model development and defines three scaling directions for the next phase.

Five Stages of LLM Evolution (2020–2026)

Tang Jie identifies five sequential stages, each driven by a different scaling mechanism:

Knowledge – Pre-training scaling: increasing data and parameter count to raise the base intelligence ceiling.

Chat – Supervised fine-tuning (SFT) and human feedback (RLHF) align models for dialogue.

Coding – Reinforcement learning (RL) becomes the primary scaling lever, fueled by verifiable rewards from benchmarks like SWE-Bench.

LHT (Long-Horizon Tasks) – Scaling extends beyond the model to environment complexity, diversity, and episode length; models must operate in richer, longer-running settings.

AAS (Autonomous Agent Systems) – Agents maintain systems, handle continuous problem streams, and attempt recursive self-improvement (RSI) of their own models, tools, and training pipelines.

This progression shows scaling shifting from static pre-training to dynamic post-training, task environments, and agent ecosystems.

Benchmark Thresholds and the KFC Framework

Tang Jie observes that when a model reaches ~80% accuracy on a domain benchmark it becomes a genuinely useful tool; at 90–95% the domain transforms and humans struggle to work without the model. He summarizes the current landscape with KFC :

K (Knowledge) – Knowledge and coding capabilities are already very strong.

F (Fighting/Attack-Defense) – Models enter real-world security scenarios; systems like Anthropic's Mythos are changing AI safety offense/defense dynamics.

C (Consumer) – Models permeate daily work and life; growing reliance on AI for routine tasks.

Two Critical Gaps

Despite progress, two gaps remain:

RSI (Recursive Self-Improvement) – The ability of a model to iteratively improve its own architecture, training data, and tooling.

Physical-world action – Robotics: perceiving, acting, and learning from feedback in the physical world.

GLM's Three Scaling Directions

Pre-training Scaling – Continue growing data and model size to lift the fundamental intelligence ceiling.

Post-training Scaling – Advance reasoning and task-solving via SFT, RLHF, and RLVR (Reinforcement Learning with Verifiable Rewards). GLM-5.3 is actively pursuing this path.

Reasoning & Agent Scaling – Expand capabilities through more complex environments, harder long-horizon tasks, and larger thinking/acting spaces.

SAO: Single-Rollout Asynchronous Optimization for Agentic RL

A July 2024 paper co-authored by Tang Jie,

Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning

, introduces SAO . Unlike GRPO's group-wise sampling, SAO uses a single rollout per update, addressing asynchrony and stability issues in long-horizon agent training. The method has already been applied to GLM-5.2 training.

In agentic RL, the model no longer merely generates answers; it continuously interacts with an environment. Consequently, the environment itself becomes part of the scaling surface.

Memory Over Context Length

Tang Jie argues that simply enlarging the context window does not equate to true long-term memory. A persistent agent must store past experiences and proactively retrieve them for new tasks. He suggests memory mechanisms may be more important than context expansion . Zhipu is currently surveying memory architectures including KV Cache, RNN/SSM, Titans, TTT, and Engram.

Toward Continuous Learning and AI Workers

If agents can continuously enter new environments, complete tasks, receive feedback, save experiences, and reuse them, they gain continual learning ability. Tang Jie predicts agents will become AI workers , initially handling digital tasks (office work, coding) and eventually advancing to scientific research — autonomously posing hypotheses, designing experiments, invoking tools, analyzing results, and iterating.

Robotics: The Next "GPT-4 Moment"

While today's models operate in digital spaces, robotics offers a complex physical environment. The perception-action-feedback loop in the real world forms a natural continual learning cycle. Tang Jie identifies robotics as a likely candidate for the next breakthrough comparable to GPT-4's impact.

In summary, GLM's journey from Knowledge through Chat, Coding, LHT, and AAS demonstrates that scaling has expanded from data and parameters to training signals, task environments, and agent systems. The next leg toward AGI moves beyond making models larger toward enabling persistent action, continual learning, and ultimately recursive self-improvement.