Why Do Large Language Models Speak and Reason Like Humans? An In‑Depth Look at Their Mechanisms

Introduction

We interact with large language models (LLMs) every day and often feel that they truly understand our language and even think like humans, despite occasional hallucinations. The article investigates what linguistic and reasoning abilities LLMs possess, how these abilities emerge from their underlying principles, methods, and internal mechanisms.

Major Claims

Higher‑order pattern learning : LLMs learn not only low‑level lexical and syntactic patterns but also high‑order semantic and world‑knowledge patterns, which distinguishes them from earlier language models.

Beyond Next‑Token Prediction (NTP) : While LLM training is framed as NTP, the overall capability results from the interaction of strategy, model architecture, optimization algorithm, and massive data.

Partial mechanistic understanding : Recent research has begun to decode LLM internals, including feature superposition, sparse autoencoders (SAE), and functional‑token memory mechanisms.

LLM Working Mechanism

LLM research can be approached from three angles: machine‑learning theory, external prompt experiments, and internal mechanistic studies. The following subsections detail the most prominent mechanistic insights.

2.1 Feature Superposition

Anthropic’s Superposition Hypothesis proposes that a single neuron can represent many more features than its count, leading to a many‑to‑many relationship between neurons and features. In a standard feed‑forward layer the computation is

where

is the input,

the weight matrix,

the bias, and ReLU is the activation function.

The hypothesis asserts that a “wide” layer with a high‑dimensional sparse feature vector can be approximated by the actual dense layer. Sparsity means that for a given input only a few features are active, reducing interference between features. For example, when processing the sentence “I visited the Golden Gate Bridge”, the sparse vector activates only a handful of features such as “Golden Gate Bridge”, “San Francisco”, “bridge structure”, and “tourist spot”.

Mathematically, the wide layer’s feature vector

is sparse, while the actual layer’s feature vector

is dense. The superposition hypothesis explains how the dense representation implicitly compresses the sparse one.

2.2 Sparse Autoencoders (SAE)

SAE is a tool for extracting interpretable features from a trained LLM. An SAE consists of an encoder that maps a residual stream

into a high‑dimensional sparse vector, and a linear decoder that reconstructs the original stream:

The training objective balances reconstruction error (making the decoded vector close to the original residual) and a sparsity regularizer that forces most dimensions of the encoded vector to be zero. After training, the encoder’s output exhibits strong sparsity, and the activated dimensions correspond to semantically meaningful concepts such as entities (“Golden Gate Bridge”) or behaviors (“sycophancy”).

2.3 Memory Mechanism – Function‑Token Hypothesis

ByteDance’s work introduces the Function Token Hypothesis : high‑frequency tokens (function words, punctuation, newline) act as memory anchors. In large‑scale pre‑training data the top 100 tokens account for roughly 40 % of all token occurrences.

Loss decomposition shows that predicting a content token from a preceding function token (“function → content”) is the slowest‑converging component, indicating that mastering this prediction drives most of the model’s optimization. Function tokens therefore learn to activate a large portion of the model’s features. Empirically, the 10 most frequent tokens activate about 70 % of all discovered features, forming a power‑law distribution.

During inference, function tokens trigger retrieval of the most predictive features. For the prompt “Answer the question in Chinese: What is the capital of Russia?”, the colon and newline activate features related to “answer in Chinese” and “Russia”, suppressing irrelevant features and guiding the model to output “莫斯科”.

These findings suggest that post‑training fine‑tuning can dramatically improve instruction following or chain‑of‑thought reasoning by reshaping the activation patterns of function tokens.

2.4 Cross‑Layer Transcoder (CLT) – Attribution Graphs

SAE captures features within a single layer, but cross‑layer interactions remain hidden. The Cross‑Layer Transcoder (CLT) learns a mapping from the residual stream of one layer to the residual streams of subsequent layers, effectively aligning feature spaces across layers.

Each CLT layer contains an encoder (non‑linear), a cross‑layer linear transformation, and a decoder (linear). The objective minimizes reconstruction error for all downstream layers while enforcing sparsity.

After training, CLT can generate an attribution graph : a directed acyclic graph where nodes represent activated features or token embeddings and edges denote significant linear influence across layers. Pruning based on activation strength and gradient‑based attribution yields a concise graph that visualizes the core feature circuits used for a specific input.

Figure 4 (from the Anthropic blog) illustrates such an attribution graph, showing how a particular feature propagates through the network to affect the final token prediction.

Language Understanding and Reasoning

LLMs have demonstrated human‑level performance on the Turing test, handling tasks that require both linguistic competence and reasoning. They can follow instructions such as “explain in English” or resolve semantic relations like “compare the Golden Gate Bridge and the Golden Arch”. Internally, high‑order patterns enable the model to combine world knowledge with syntax, as evidenced by extracted features for entities and behaviors.

Nevertheless, LLMs differ fundamentally from the human brain. Human language processing involves specialized regions (Broca’s and Wernicke’s areas) and embodied cognition, whereas LLMs rely purely on statistical pattern learning and transformer architectures. Consequently, LLMs suffer from hallucinations—outputs that are factually incorrect—because the training objective optimizes likelihood, not truth. Retrieval‑augmented generation (RAG) is one practical mitigation.

Comparison with Human Abilities

Table 1 (adapted from the article) contrasts LLM capabilities with human abilities. LLMs excel in pure language and reasoning tasks, sometimes surpassing humans, but they lack multimodal perception, embodied experience, formal logical rigor, and consciousness. Their “thinking” is a heuristic generation process rather than symbolic reasoning or conscious deliberation.

Hallucinations arise from the probabilistic nature of next‑token prediction; they cannot be eliminated solely by scaling. Multimodal LLMs (MLLMs) begin to integrate vision or audio, yet their reasoning still occurs in a language‑only latent space, far from the embodied, sensorimotor loops that underlie human thought.

Creativity in LLMs appears limited to incremental (interpolative) innovation; truly disruptive (extrapolative) breakthroughs such as formulating new scientific theories remain unproven.

Conclusion

The article synthesizes recent advances that demystify how LLMs acquire language‑like and reasoning‑like behavior: feature superposition compresses a vast sparse representation; sparse autoencoders expose interpretable concepts; function tokens act as memory anchors; and cross‑layer transcoders reveal the flow of information across the network. While these insights bring LLMs closer to mechanistic understanding, significant gaps remain regarding hallucination mitigation, embodied cognition, logical rigor, and consciousness.