When Does Trajectory Review Boost Agent Success? Five Key Factors Explained

Problem definition

Agent trajectory review can be categorized along three dimensions: (1) when the review occurs, (2) what evidence it uses, and (3) how the output is applied. The three layers are response‑level self‑correction (immediate post‑answer revision), cross‑episode (post‑task) reflection (store failure attribution and inject into the next attempt), and in‑task reflection with dynamic replanning (state‑goal alignment at each step).

Positive evidence

Response‑level self‑correction

Renze & Guven (2024) evaluated nine LLMs on 1,000 multi‑domain multiple‑choice questions. After a wrong answer they generated eight types of self‑reflection prompts (Retry, Keywords, Advice, Instructions, Explanation, Solution, Composite) and answered again. All prompt types produced statistically significant gains (p < 0.001). For GPT‑4 the baseline accuracy 0.79 rose to 0.83 with Retry, 0.84 with Keywords, 0.85 with Advice, 0.86 with Instructions, 0.88 with Explanation, 0.93 with Solution, and 0.93 with Composite. More concrete, plan‑oriented prompts (Advice/Instructions/Solution) yielded larger improvements than vague keywords or simple failure notices.

Cross‑episode (post‑task) reflection

Reflexion stores a natural‑language failure attribution after an episode and injects it into the next attempt without updating model weights. Reported results: HumanEval pass@1 = 91 % (GPT‑4 baseline = 80 %). Subsequent works extended this line:

ExpeL retrieves similar successful trajectories to avoid re‑trying, achieving 40 % on HotPotQA (comparable to Reflexion’s 39 % with three retries) without additional attempts.

Retroformer trains a dedicated review model with policy‑gradient optimization, yielding continuous improvement across multiple tasks.

Agent‑Pro adds policy‑level reflection for strategic games, surpassing both original LLMs and specialized game models.

All these successes share low‑noise external signals (unit tests, environment rewards, tool verification).

In‑task reflection and dynamic replanning

ReflAct (2025) modifies ReAct by comparing the current state to the goal at every step. On ALFWorld GPT‑4o success rises from 85.1 % (ReAct) to 93.3 % (+27.7 % relative). On Llama‑3.1‑8B the relative gain is +126.9 % and no new failure modes appear, indicating that step‑wise state‑goal alignment is more fundamental than post‑episode attribution.

Plan‑and‑Act isolates dynamic replanning. On WebArena‑Lite static planning + synthetic data raises success from 9.85 % to 43.63 %; adding per‑step replanning adds another 10.31 pp to 53.94 %, and with chain‑of‑thought reaches 57.58 % (SOTA). Even with an un‑fine‑tuned executor, providing high‑quality dynamic plans lifts success from 9.85 % to 44.24 %.

Negative evidence and boundary conditions

Intrinsic self‑correction without external feedback

Huang et al. (ICLR 2024) reproduced intrinsic self‑correction (model checks and edits its own reasoning) on GSM8K and CommonSenseQA. Accuracy dropped for GPT‑4‑Turbo (91.5 % → 88.0 % after one correction) and GPT‑3.5 often changed correct answers to wrong ones. The authors note that prior positive reports relied on oracle labels (knowing the correct answer) to stop correction; removing this cue eliminated gains. Follow‑up work shows LLMs struggle to locate reasoning errors but can correct them when the error location is provided, and that self‑generated candidate selection does not outperform generation.

Task and environment constraints

Reflexion’s authors observed a failure case on WebShop: after four reflection rounds success fell from 43 % to 35 %. Analysis revealed that success depended mainly on the quality of the search‑engine results, not on the agent’s planning ability, showing that reflection cannot compensate for bottlenecks in external tools.

Reproductions of Reflexion on HotPotQA reported a baseline ReAct score of 0.09 instead of the paper’s 0.26, highlighting high sensitivity to prompt and implementation details.

Five modulating variables

Verifiability of feedback : Low‑noise external signals (unit tests, environment success flags, tool verification) are essential for sizable gains.

Timing of review : Post‑episode attribution is simple and suits retryable tasks; in‑task step‑wise reflection yields larger gains for long‑horizon tasks; per‑step replanning adds the biggest boost but incurs higher token cost.

Specificity of review output : Concrete next‑step instructions or solutions outperform vague advice or mere failure notices.

Backbone capability : Stronger models locate errors better and produce useful reflections; weak backbones may generate hallucinated or harmful feedback.

Cost‑benefit trade‑off : Multiple correction rounds multiply token usage; dynamic replanning should be triggered only on state‑goal mismatches; experience quantity matters less than relevance.

Practical implications for agent engineering

Provide reliable external feedback (unit tests, tool validators, environment rewards) before adding reflection. When feedback is absent, reflection tends to be net negative (Huang et al. 2024).

Design reflection outputs to be actionable—concrete plan revisions, executable instructions, or step‑by‑step solutions (Renze & Guven 2024).

Prioritize lightweight state‑goal alignment during execution (ReflAct) before resorting to post‑episode attribution.

Distill both successful and failed trajectories into reusable strategies; ReasoningBank shows that mixing failure examples adds value, whereas AWM that only stores successes can degrade performance when mixed with failures.

Implement stop‑loss mechanisms: limit reflection rounds, use confidence or external verification to accept edits, and guard against self‑bias that inflates correct answers.

Diagnose the bottleneck before deciding on reflection. If failures stem from tool quality (e.g., WebShop), improving the tool is more effective than adding reflection.

When feasible, train a dedicated reflection module (e.g., RISE, Retroformer). Fine‑tuned modules show more stable gains and scale with model size.

Conclusion and outlook

Agents can improve task success rates by reviewing their trajectories, but only when reliable external feedback, concrete actionable outputs, appropriate timing, and sufficient model capability are present. The convergent findings of recent papers indicate that feedback quality—not the specific form of review—determines effectiveness. Future work is shifting from prompt‑based tricks toward trainable self‑improvement modules, error‑location detectors, failure‑trajectory distillation, and selective replanning strategies.

Representative results (selected works)
------------------------------------------------------------
Work (Year)          | Review timing | Feedback source                | Review output                | Key result
---------------------|--------------|------------------------------|-----------------------------|------------------------------
Reflexion (2023)     | post‑episode | env success / unit tests      | failure attribution → memory| HumanEval 91% pass@1 (GPT‑4 80%)
Self‑Refine (2023)   | post‑answer  | model self‑feedback           | feedback + rewrite          | ~20% absolute gain on 7 tasks (baseline disputed)
CRITIC (2023)        | post‑answer  | tool interaction verification| tool‑validated correction    | consistent gains on QA / math / detox
Renze & Guven (2024) | post‑episode | correct answer (oracle)       | 8 reflection types          | GPT‑4 0.79→0.93 (Composite), p<0.001
ExpeL (2023)         | offline cross‑task| success & failure trajectories| insight library + retrieval | HotPotQA 40% (no retries)
Retroformer (2023)  | post‑episode | env reward (policy gradient)  | learned reviewer model      | continual multi‑task improvement
RISE (2024)         | iterative    | optional env feedback         | recursive self‑correction   | GSM8K LLaMa‑3‑8B +8.2%, scaling with size
ReflAct (2025)       | per‑step    | state‑goal observation        | goal‑anchored reflection    | ALFWorld GPT‑4o 85.1%→93.3% (+27.7% rel.)
Plan‑and‑Act (2025)  | per‑step    | env observation               | rewrite remaining plan      | WebArena‑Lite +10.31pp (total 57.58% SOTA)
LATS (2023)          | per‑step search| env feedback + self‑reflection| MCTS trajectory search      | HumanEval 94.4% pass@1 (GPT‑4), WebShop +22.1
Tree search (Koh 2024)| per‑step search| env + value function          | best‑first search + backtrack| VisualWebArena +39.7% (26.4%→SOTA)
AWM (2024)           | offline/online| successful trajectories       | reusable workflow library   | Mind2Web +24.6%, WebArena +51.1%
ReasoningBank (2025) | online       | LLM‑as‑judge success/failure | distilled reasoning strategy| WebArena +4.6~8.3pp, SWE‑Bench‑V 34.2%→38.8%
ACE (2025)           | online       | execution feedback            | evolving playbook (generate‑reflect‑curate) | AppWorld +10.6pp
Voyager (2023)      | in‑task + cross‑task| env feedback / self‑validation| executable skill library   | 3.3× unique items, 15.3× tech node unlock speed
------------------------------------------------------------