Skip to content
SLS Session Store: The Data Foundation for Agent Recursive Self-Improvement

SLS Session Store: The Data Foundation for Agent Recursive Self-Improvement

Alibaba Cloud SLS Session Store transforms OpenTelemetry traces and logs into standardized execution trajectories, enabling continuous data accumulation for Agent observability, dataset construction, LLM-based evaluation, and recursive self-improvement (RSI) loops, as demonstrated through a coding agent debugging case study.

Alibaba Cloud Observability
Alibaba Cloud Observability
Alibaba Cloud Observability

Agent Self-Evolution Requires Continuous Execution Data

Recursive Self-Improvement (RSI) aims to let systems use their own execution and evaluation feedback to produce lasting capability improvements, then feed the improved system into the next self-improvement cycle. For AI agents, this means prompts, tools, task strategies, and even the improvement process itself become objects of continuous optimization. Each execution round — what the agent did, what feedback it received, how it performed after changes — must leave a trace usable by the next round.

From Scattered Traces to Standardized Trajectories

Alibaba Cloud SLS (Simple Log Service) is a cloud-native data platform for logs, metrics, and traces. As AI applications evolve from single-turn Q&A to agent-driven multi-step tasks, the need arises to organize scattered model calls, tool executions, and feedback into readable, analyzable execution processes. SLS Session Store parses and correlates incoming OpenTelemetry traces and logs into standardized Trajectories (execution trajectories), combining task inputs, model/tool interactions, results, and runtime metrics. These trajectories are designed for both human reading and programmatic/LLM analysis.

Data Perspectives: Raw, Trajectory, Session, Feature

Raw records : What runtime facts were collected and their sources.

Execution Trajectory : What the agent did, what feedback it received, what outputs and costs were produced.

Task Session : How multi-turn interactions continue the same task context under explicit session and turn associations.

Feature Data : Which behaviors and statistics can be used for filtering, analysis, and subsequent improvement.

This common trajectory representation gives downstream evaluation and self-evolution systems a unified starting point, reducing the work of re-interpreting record formats and stitching together execution flows.

Accumulating and Querying Data in SLS

Teams can continuously accumulate trajectories in SLS, using its query and analysis capabilities to locate tasks by inputs, outputs, models, tools, latency, or token usage, then read specific actions and feedback, and fall back to raw records for fact-checking. SLS data consumption interfaces also allow downstream applications to stream trajectories for ongoing processing.

Case Study: Debugging a Pagination Filter Fix

A coding agent fixing a pagination filter issue searched code, modified implementation, ran tests, and continued acting on feedback. The trajectory reveals: the agent identified that page requests didn't preserve filter conditions; after modification, component tests passed, but browser verification failed due to missing runtime dependencies. The agent retried the same test command repeatedly, still getting startup errors, yet summarized as "test verification passed." In Session Store, developers can trace along test calls to see returns, combine with subsequent actions and final output: component tests had results, browser verification remained incomplete; retries added no new verification info, and the final reply didn't acknowledge the gap. This yields concrete improvement directions: add environment checks, adjust retry strategies, accurately report verification scope. Successful modifications and tests are also recorded for reference. Resource analysis similarly benefits from process data — combining call content, results, latency, and token usage to judge which consumptions advanced the task and which were blocked by repeated conditions.

Building Reusable Datasets from Daily Executions

Session Store provides a queryable, traceable data source for sample construction. Teams can select typical issues, high-frequency tasks, and reliably completed cases, verify and categorize them by business goals, and supplement evaluation criteria. For the fix trajectory, they can retain task requirements, code and environment context, test feedback, and subsequent actions as an evaluation sample for environment anomaly handling. Reliably completed similar fixes can be included to cover both normal and difficult scenarios. Full tasks suit overall completion checks; key segments suit next-action research. Maintaining links between samples and original trajectories enables context review and continuous dataset augmentation.

LLM-as-a-Judge: Evaluating Results and Processes

Standardized trajectories enable LLM-as-a-Judge , where a large model acts as evaluator. Teams provide task requirements, execution data, and business scoring rubrics, optionally with reference answers or expected outcomes, and ask the model to analyze result quality and action processes, producing scores and rationales. In the fix task, evaluation can focus on whether the agent understood tool errors, took reasonable actions, and accurately reported verification scope. Open tasks can assess tool selection, evidence grounding, and task progression, preserving space for different valid completion paths. Downstream tools can batch-analyze daily run quality, flag tasks needing attention, and compare versions after model, prompt, or tool changes by running the same task sets and comparing results, trajectories, and costs. Repeated runs on critical tasks reveal stability. Code fix effectiveness can be checked via actual tests; business rule satisfaction via rule validation; human spot-checks calibrate LLM judgments against team standards. These feedback signals help the self-evolution system prioritize issues and choose among candidate improvements.

The RSI Loop: Continuous Improvement Powered by Shared Data

For RSI, execution data feeds a continuous cycle: execute tasks → analyze feedback → generate candidate improvements → verify and select → update agent → repeat with updated system. Experience summarization seeds candidate proposals; the self-evolution system must turn proposals into retainable changes and verify them on new tasks so the next round starts from validated capabilities. Session Store continuously supplies run data: which tasks revealed problems, which actions candidates changed, what feedback the new round produced — all analyzable around trajectories.

Generating Concrete Improvements from Run Feedback

In the missing-dependency example, the downstream self-evolution system can aggregate multiple similar trajectories to induce which conditions frequently block verification, then generate distinct candidate fixes: add environment checks to task strategy, augment tools with dependency diagnostics, or adjust prompts so final replies clearly distinguish verified from blocked portions. Improvements target the agent's actual working methods — prompts, tool code, execution strategies. Teams can iterate on one high-frequency issue first, then expand to more complex tasks. Session Store trajectories provide context for these proposals; both successful and blocked processes become analysis material, helping the system understand when a tactic applies and reducing re-proposal of already-tried approaches.

Verifying with New Tasks and Selecting Retained Changes

After candidate generation, the self-evolution system can run different versions on the same task suite, combining actual tests, LLM evaluation, and runtime costs to compare performance and select goal-meeting improvements. For the environment-check proposal, verification must cover both dependency-present and dependency-missing tasks: can normal tasks still complete? Do blocked tasks get effective diagnostics? Are retries reasonable? Is the final report accurate? Testing on tasks not used for proposal generation helps judge generalization. Linking run trajectories with agent versions, candidate changes, and evaluation results enables tracing the basis of each selection. Unsuccessful attempts — those yielding no gain, harming other tasks, or significantly increasing cost — are also worth recording for future reference.

Letting the Improved System Participate in Further Improvement

Validated changes become the new version for subsequent executions. New traces and logs flow back into Session Store, forming new standardized trajectories that continue to feed evaluation and self-evolution systems. Subsequent iterations can address new problems: now that environment checks are added, are tool feedbacks still ignored? After reducing retries, does premature verification termination appear? The system can combine old and new trajectories to propose the next batch of candidates and re-verify effects. Under comparable task goals and evaluation conditions, continuously accumulated data lets teams observe multi-round change processes, understand where gains come from and their applicability scope, and retain effective versions while re-examining unselected alternatives.

Meta-Improvement: Making the Agent Better at Improving Itself

A deeper RSI exploration lets the system modify its own improvement-generation methods — e.g., adjusting how problem samples are selected, refining prompts for failure-cause analysis, or optimizing candidate generation and experiment scheduling. The improved mechanism then produces new improvements, forming a deeper recursion. Here, the improvement process's own run records can also be collected; once they conform to the ingestion convention and enter Session Store, downstream systems gain data to analyze which experiments yielded useful information and which search paths deserve more investment. This requires both task execution and improvement processes to leave traceable evidence and preserve independent effect verification throughout changes. Session Store's data organization and reuse capabilities provide the foundation for gradually building such a process.

Conclusion

From a single execution's traces and logs, to queryable standardized trajectories, to samples, evaluation feedback, and multi-round improvements, run data can continuously participate in the evolution of agent capabilities. Alibaba Cloud SLS Session Store, backed by SLS storage and analysis, organizes and supplies execution data; downstream systems like AgentLoop can build evaluation, generate candidate improvements, and verify effects around this data. Teams can start from concrete tasks and progressively incorporate more feedback, selection, and update stages into the self-evolution loop. As improved agents continue executing tasks, producing feedback, and joining the next improvement round, continuously accumulated data becomes the foundation connecting each iteration cycle.

Agent RSI loop diagram
Agent RSI loop diagram

Related documentation: Alibaba Cloud SLS product page (https://www.aliyun.com/product/sls) and SLS official documentation (https://help.aliyun.com/zh/sls/what-is-log-service).

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

ObservabilityOpenTelemetryDataset ConstructionAlibaba CloudLLM EvaluationRecursive Self-ImprovementAgent Self-EvolutionRSIExecution TrajectoriesSLS Session Store
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.