Why LLMs Miss Simple Addition: Geometric Mechanism Behind Arithmetic Errors

A recent ICML 2026 paper from Nanjing University reveals that large language models encode correct arithmetic information in structured geometric manifolds, yet errors arise from noisy quantization at decision boundaries, and proposes probing, Iso‑Raw‑Sum Trajectory, and a dual‑stream consistency check to diagnose and correct these mistakes.

Arithmetic ErrorsDual-Stream ConsistencyIso-Raw-Sum Trajectory

0 likes · 11 min read

Why LLMs Miss Simple Addition: Geometric Mechanism Behind Arithmetic Errors

Machine Heart

Jun 17, 2026 · Artificial Intelligence

Why Large Language Models Miss Simple Addition: Iso‑Raw‑Sum Trajectories Reveal the Geometry of Errors

Despite excelling at complex reasoning, LLMs often err on multi‑digit addition; probing shows correct answers reside in hidden states, and the authors reveal a structured geometric manifold—digit basins, carry fibers, and Iso‑Raw‑Sum trajectories—explaining how errors arise via noisy quantization at decision boundaries.

Arithmetic ErrorsGeometric AnalysisLLM

0 likes · 12 min read

Why Large Language Models Miss Simple Addition: Iso‑Raw‑Sum Trajectories Reveal the Geometry of Errors

Network Intelligence Research Center (NIRC)

Nov 11, 2025 · Artificial Intelligence

What Is Mechanistic Interpretability and Why It Matters for Large Language Models

The article defines mechanistic interpretability as reverse‑engineering LLMs to reveal how they represent knowledge and make decisions, explains its importance for transparency, risk mitigation, and model improvement, and surveys key techniques such as causal tracing, zero‑making, noise‑making, and logit‑lens methods with illustrative examples.

Mechanistic Interpretabilitycausal tracinglarge language models

0 likes · 8 min read

What Is Mechanistic Interpretability and Why It Matters for Large Language Models