Machine Learning Algorithms & Natural Language Processing

Jul 9, 2026 · Artificial Intelligence

How Much Can Large Language Models Remember? ICML 2026 Finds ~3.6 bits per Parameter

An ICML 2026 award paper quantifies the memory capacity of GPT‑style language models, showing that each parameter stores roughly 3.6 bits of information, and explores how this capacity scales with model size, data volume, precision, and its impact on generalization and privacy risks.