How OpenClaw‑RL Turns Everyday Interactions into Self‑Evolving AI

Extracting Hidden Interaction Signals

The research team observed that every action of an intelligent agent generates a massive stream of feedback from the environment, including user replies and tool outputs. Traditional systems treat this information merely as context for the next response and discard it afterward. OpenClaw‑RL classifies the discarded interaction data into two distinct dimensions: evaluative signals (natural scores indicating success or failure) and instructional signals (explicit guidance for correction).

Evaluative signals act as a built‑in scorer: a correct test case receives a positive score, a failed execution receives a negative score, and ambiguous interactions receive zero. Instructional signals provide richer direction, such as a user pointing out the misuse of a specific library or an error log that reveals the exact logical flaw.

Decoupled Asynchronous Mechanism

To capture these signals without affecting user experience, the system is built on the open‑source asynchronous framework slime. Four independent loops run concurrently: model inference service, environment execution node, reward scoring system, and strategy‑training engine. Each loop operates without waiting for the others, enabling seamless real‑time data collection.

The process reward model continuously analyzes incoming dialogues, while the Megatron training engine updates model weights based on accumulated gradients. All interaction data and reward scores are logged asynchronously, ensuring zero added latency. Logs are cleared after each weight update to keep data aligned with the current policy version.

Merging Scoring and Text Guidance

OpenClaw‑RL employs two complementary mechanisms:

Binary reinforcement learning handles evaluative signals. A majority‑vote judge model assigns +1, 0, or –1 to each action based on the subsequent environment state.

Online policy distillation processes instructional signals. When a user’s reply contains a clear correction, the system extracts a token‑level supervision instruction, filters out short or low‑information prompts, and appends the instruction to the next teacher context.

The judge model may issue multiple independent queries and aggregate results by majority vote. The distilled instructions are used to bias token probabilities during a second pass, giving positive advantage to correct tokens and suppressing incorrect ones.

Real‑World Performance Validation

The team evaluated the framework on two parallel tracks. In the personal‑agent track, a simulated student used a private device to write assignments while trying to hide AI assistance. The assistant learned from 36 interactions and produced human‑like responses. In the teacher‑feedback track, a model acted as a strict teacher, providing detailed, friendly comments after only 24 interactions.

Both tracks leveraged a matrix of models ranging from 40 B to 320 B parameters, running on cloud‑scale parallel hardware. The hybrid approach (binary RL + online distillation) consistently outperformed pure result‑only reward methods, achieving higher accuracy on long‑horizon tasks thanks to the process reward model’s step‑wise scoring.

Overall, OpenClaw‑RL demonstrates that continuously harvested interaction data, when properly split into evaluative and instructional signals and processed through an asynchronous, decoupled pipeline, can drive self‑evolving agents that improve with every user exchange.