Can AI Learn Mental Math? Implicit Chain‑of‑Thought Proven Theoretically (Stuart Russell)

Recent AI reasoning models incur high token costs because explicit Chain‑of‑Thought (CoT) generates hundreds to thousands of intermediate "thinking" tokens for each inference step. These tokens act as a visible scratchpad, inflating compute resources by more than tenfold for complex math problems.

The fundamental limitation is structural: as long as each intermediate step is emitted as a token, inference latency has a lower bound proportional to the length of the reasoning chain.

The paper Transformers Provably Learn to Internalize Chain‑of‑Thought (arXiv:2605.28600v1) from UC Berkeley and Princeton introduces Implicit Chain‑of‑Thought (ICoT) , which asks whether a model can hide the intermediate steps inside its hidden states and output only the final answer.

Earlier work (Yuntian Deng et al., 2024) proposed a step‑wise masking approach that gradually hides tokens, but it required a training stage for each step, leading to linear training overhead with the number of reasoning steps.

Core Innovation: Tree‑Structured Curriculum

The authors observe that the k‑parity problem—a classic theoretical benchmark—can be represented as a binary tree of depth log₂k. Each internal node computes the product of its two child bits, culminating at the root.

Standard ICoT hides one token per stage, ignoring this hierarchy. The new Log‑ICoT curriculum hides an entire tree layer at once, reducing the number of stages from k‑1 to log₂k (e.g., from 15 to 4 stages when k=16).

This alignment lets each Transformer layer specialize in processing one level of the tree, matching model architecture to the reasoning structure.

Theoretical Guarantee

Theorem 1 states that an L‑layer Transformer trained with Log‑ICoT on the k‑parity task requires only polynomial‑size samples (≈n^(2+ε)) and log₂k gradient steps to predict the correct parity with probability approaching 1, achieving exponentially small error.

The proof overcomes two technical challenges:

Representation Collapse: Deep layers tend to homogenize token representations, diluting gradients. The authors introduce gated connections that activate only the positions corresponding to the current tree level, preserving distinct gradients.

Error Propagation: Small approximation errors in early stages can amplify later. After each gradient update, attention weights are quantized to the nearest integer, effectively locking previously learned layers and preventing error accumulation.

Empirical Validation

Experiments on n=30 input bits with k=16 (four Transformer layers, four training stages) achieve 100 % validation accuracy. During training, loss drops to near zero in the first stage (full CoT visible). Each subsequent stage replaces half of the remaining CoT positions with zeros, causing a brief loss spike that quickly recovers as the model assimilates the new constraint.

By the final stage, all CoT positions are zeroed, yet the model still predicts parity perfectly. Attention heatmaps show each layer focusing on the corresponding tree level, confirming the theoretical alignment.

Conclusion

The work fills a theoretical gap by proving that implicit reasoning can be learned under clear conditions, bridging the divide between empirical success of ICoT and understanding of why it works. It suggests a path toward compressing long reasoning chains into hidden states, eliminating costly intermediate token generation, though extending the approach to real‑world LLMs will require handling tasks without explicit hierarchical structure.