The author tests CodeX and GPT5.6‑Sol on a multi‑task directory workflow and finds simple yet puzzling errors, then observes Fable5 failing on basic CSS tweaks, linking both issues to catastrophic forgetting and hallucination in large language models.
The author tests CodeX, GPT5.6‑Sol and Fable5, exposing simple yet puzzling errors—missed tasks, inconsistent CSS changes, and contradictory answers—that may stem from catastrophic forgetting and highlight emerging reliability bottlenecks in large language models.
Google researchers propose a ‘sleep’ stage for large language models, where after active use the model consolidates recent experiences through knowledge‑seeding and self‑generated ‘dreaming’ tasks, addressing catastrophic forgetting and enabling continual learning.
The paper analyzes lifelong model editing, showing that Lifelong Normalization (LN) is essential for preventing catastrophic forgetting and model collapse, explains the positive cumulative effect of early edits, and introduces StableEdit with warm‑up and full whitening to achieve robust, million‑scale editing.
Agent-Dice introduces a geometric consensus filtering and curvature‑based importance weighting framework that disentangles knowledge updates, preventing catastrophic forgetting in large‑language‑model agents while enhancing plasticity, and demonstrates superior stability‑plasticity trade‑offs on GUI and tool‑use benchmarks across multiple base models.
The article analyzes why standard prompt tuning (CoOp) causes catastrophic forgetting in visual‑language models, introduces the KgCoOp framework that adds a knowledge‑guided loss to regularize prompts, and shows through extensive experiments on 11 benchmarks that KgCoOp improves unseen‑class accuracy, harmonic mean, and efficiency while discussing trade‑offs and limitations.
The article explains On-Policy Distillation, derives its forward and reverse KL gradients, introduces Self‑Distillation where the policy serves as its own teacher, discusses practical implementation tricks such as extra‑knowledge injection, EMA or trust‑region teacher stabilization, and highlights benefits like reduced catastrophic forgetting, fewer Aha moments, and a narrower train‑test gap, especially for larger models.
This article analyzes the GraphKeeper framework, which combines multi‑domain graph decoupling, unbiased ridge‑regression knowledge preservation, and a domain‑aware distribution discriminator to overcome catastrophic forgetting in domain‑incremental graph neural network training, and validates its superiority through extensive experiments and ablations.
This article critically examines recent RL‑for‑LLM studies, revealing that reinforcement learning improves search efficiency but does not extend the intrinsic reasoning capabilities of base models, and explores the underlying model‑conditioned optimization bias, comparisons with SFT distillation, and the trade‑off with catastrophic forgetting.
The article analyzes why reinforcement learning (RL) fine‑tuning retains a large language model's general abilities better than supervised fine‑tuning (SFT), explaining the off‑policy distribution shift of SFT and the on‑policy data consistency, KL penalty, and trust‑region mechanisms that give RL its anti‑forgetting properties.
This paper proposes a multi‑teacher distillation framework for continual learning that combines active data rehearsal with feature‑decoupled distillation, demonstrating superior performance on PASCAL VOC and COCO benchmarks while mitigating catastrophic forgetting and balancing stability‑plasticity trade‑offs.
Continuing pretraining (CP) bridges pretraining and SFT to inject domain knowledge, but faces catastrophic forgetting; this article explores leveraging importance sampling to balance common and domain data, discusses data selection, annealing strategies, and practical tips for mitigating forgetting while enhancing specialized capabilities.
