What Do Large AI Models Actually Learn During Pre‑training?

Common Misconception

When asked what large‑model pre‑training learns, many engineers answer that the model simply memorizes knowledge from the corpus, as if it were reading the entire Internet. This view cannot explain why models answer unseen questions, generate novel code, or exhibit sudden capability jumps as parameters increase.

Ilya Sutskever, former chief scientist at OpenAI, famously said that predicting the next word is essentially compressing the world. This insight frames the rest of the analysis.

Layer 1: Data Compression – The Real Goal of Pre‑training

Mechanistically, pre‑training asks the model to predict the next token given the preceding text, using the text itself as the supervision signal (self‑supervised learning). To succeed, the model must discover patterns that enable high‑compression of the input.

Imagine compressing a million‑word novel. Storing each character yields zero compression, but recognizing that certain characters rarely follow others, that protagonist names recur, or that martial‑arts descriptions follow fixed templates dramatically reduces storage size. The deeper the pattern discovery, the higher the compression ratio.

Large models perform the same operation: billions of tokens are the input, model weights become the compressed artifact. To minimize prediction error, the model must learn all statistical structures behind the text—lexical usage, code bracket matching, factual statements, and reasoning patterns. Thus the model does not merely “store knowledge”; it stores a compressed encoding of the world’s statistical structure.

Consequently, facts that appear frequently are remembered well, while rare facts are prone to forgetting or hallucination, because compression inevitably involves trade‑offs.

Layer 2: Statistical Regularities & Generalization – Parrot or Understanding?

Critics label large models as “stochastic parrots” that only mimic surface patterns. This was plausible for early GPT‑2 models but no longer holds. The key evidence is their generalization ability.

Models can solve unseen math problems step‑by‑step and translate low‑resource languages by leveraging cross‑lingual patterns, indicating they have inferred underlying generative mechanisms rather than merely memorized examples.

When compression pressure is high, memorizing every sample becomes impossible; the only viable strategy is to infer the data‑generation process. Analogously, two students study history: one memorizes pages, the other abstracts a causal framework. The latter can answer questions about unseen material, just as large models must learn the generative rules of language.

Nevertheless, the model’s “understanding” is limited to textual statistics; lacking embodied experience, it can make bizarre errors that are not due to stupidity but to the one‑dimensional nature of its training data.

Layer 3: Representation Learning – Semantic Vectors in High‑Dimensional Space

The most subtle outcome of pre‑training resides in the intermediate layers: each token and sentence is mapped to a point in a high‑dimensional vector space whose geometry reflects meaning.

Classic examples include the vector arithmetic "king – man + woman ≈ queen," where gender and royalty form orthogonal directions, synonyms cluster together, and causal relations acquire their own geometric structure.

This emergent representation arises without explicit linguistic supervision; the compression pressure forces the model to encode semantic relationships as geometric relations.

Two practical implications follow for engineers:

Embedding retrieval works because documents and queries share the same semantic space; similarity becomes a distance problem.

Semantic gaps appear when the training corpus does not reflect a specific domain’s terminology, leading to mismatched vectors that require domain‑specific fine‑tuning or hybrid retrieval.

Layer 4: Emergence – Phase Changes Triggered by Scale

Some abilities are absent in small models but appear abruptly once parameters cross a threshold, defying smooth extrapolation. Examples include three‑digit addition, chain‑of‑thought reasoning, and planning.

One prevailing hypothesis is that complex abilities are composites of many sub‑abilities that each improve smoothly with scale; only when all sub‑abilities surpass a competence line does the composite skill manifest, akin to a phase transition in physics.

From an engineering standpoint, this means capability boundaries are not predictable from size alone; empirical evaluation on the target task is essential.

For instance, a 13‑billion‑parameter model outperformed a 1.6‑trillion‑parameter model on an Agent benchmark after fine‑tuning, showing that the “last subject” can be pushed over the passing line.

Pre‑training vs. Downstream Skills: Foundations and Techniques

Pre‑training, instruction fine‑tuning (SFT), and reinforcement learning from human feedback (RLHF) each contribute differently. The widely accepted view is that core abilities form during pre‑training and are merely “awakened” during alignment, not taught from scratch.

Using a martial‑arts metaphor: pre‑training builds internal power (reading countless texts), SFT teaches specific forms (how to respond to prompts), and RLHF refines performance in the arena (human‑preferred responses).

Evidence shows that a base model already possesses reasoning, translation, and coding potential; without alignment it tends to continue the input rather than serve as an assistant. Alignment redirects existing knowledge toward helpful behavior.

Practical guidance emerges:

If a task requires the model to know new facts, retrieval‑augmented generation (RAG) is necessary because prompting or fine‑tuning cannot inject large amounts of private knowledge.

If a task demands output style, format, or constraints, fine‑tuning is the appropriate tool.

Confusing these layers leads to misallocation of AI project budgets.

Conclusion: Compression Is Intelligence, But Not the Whole Story

Answering the opening question, the model learns to predict the next token, which results in lossy compression of the textual world; this compression yields a structured semantic space and a set of generalizable regularities; at sufficient scale, emergent abilities such as reasoning and creation appear.

Ilya Sutskever’s claim that “compression is intelligence” captures the core but should not be idolized. Compression gives the model knowledge; alignment gives it usability. Engineers must still handle hallucinations, knowledge freshness, and domain adaptation through RAG, tool use, and observability.

Teams that understand what the model learns tend to focus effort on what the model cannot learn—private data, business rules, and safety guards—making their AI applications more robust.