How Large Language Models Can Self‑Improve: A Technical Review and Future Outlook

Introduction

Large language models (LLMs) have achieved rapid performance gains by scaling model size, data, and compute (Brown et al., 2020; Ouyang et al., 2022; Hoffmann et al., 2022; OpenAI et al., 2024). The prevailing assumption is that larger models trained on higher‑quality, human‑annotated data will become stronger. In practice, methods such as RLHF rely heavily on carefully curated supervision signals, but this paradigm faces two structural limits: (1) the scarcity and rising cost of high‑quality human data (Gilardi et al., 2023; Villalobos et al., 2024); and (2) the cognitive ceiling of humans, which reduces the informative gradient when models approach or surpass human performance (Bowman, 2023; Burns et al., 2023). These pressures motivate research into model self‑improvement, where the model generates data, evaluates outputs, and iteratively refines itself without continuous human‑in‑the‑loop supervision.

Self‑Improvement Definition

We define LLM self‑improvement as a learning paradigm that operates without persistent human supervision, featuring two core attributes:

Autonomy : the improvement loop runs without ongoing manual labeling or correction, though external components such as teacher models, validators, critics, reward models, or automatic evaluators may still be employed.

Continuity : improvement is an iterative, self‑reinforcing process where each round re‑uses earlier outputs to generate stronger supervision signals, enabling cumulative progress over time.

This view frames self‑improvement not merely as a performance boost but as a structural capability for sustained, autonomous growth.

Closed‑Loop Lifecycle Architecture

Guided by the vision of an autonomous self‑improvement system, we propose a lifecycle consisting of five tightly coupled components (see Figure 2):

Data Acquisition : the model autonomously collects or synthesizes training data.

Data Selection : the model independently evaluates and filters data points to retain higher‑quality, more informative examples.

Model Optimization : the model learns from the selected data, updating its parameters to enhance capability.

Inference Refinement : during inference the model improves its output without altering underlying parameters.

Autonomous Evaluation : a continuous feedback mechanism that monitors performance and guides subsequent improvement cycles, addressing the rapid obsolescence of static benchmarks.

These components place the model at the core of an automated iterative loop, ensuring that improvement signals are generated, screened, applied, refined, and evaluated in a consistent manner.

Related Work and System‑Level Perspective

Recent surveys have examined self‑improvement from specific angles: Tao et al. (2024) focus on self‑training and reinforcement learning for policy‑level evolution; Dong et al. (2024) review prompting and decoding‑time refinement techniques; Fang et al. (2025a) and Gao et al. (2026) emphasize agent‑based systems with memory, reflection, and tool‑enhanced interaction. Most prior work concentrates on isolated stages (e.g., training or inference). In contrast, our system‑level view treats self‑improvement as a unified closed‑loop lifecycle that integrates all stages of model development into a scalable, end‑to‑end framework.

Paper Organization

The remainder of the paper is split into two parts. Sections 2–6 systematically examine each component of the self‑improvement system, providing high‑level overviews followed by structured categorizations of representative methods (illustrated in Figure 3). Each section ends with a discussion summarizing key insights and how the component interacts with the others. Sections 7–9 broaden the discussion to cover challenges, limitations, applications, and future research directions, including work on self‑evolving agents (e.g., Section 5.4 on agent‑based inference refinement) and cross‑domain use cases.