RSIGym: Open-Source RSI Research Environment for Joint Model-Harness Optimization

Why RSI Is a Systems Engineering Problem

An AI agent that recursively improves itself must generate data, train models, run evaluations, and iterate. The authors argue that Recursive Self-Improvement (RSI) is not merely a model problem but a systems engineering challenge: a model's failure on a task can stem from missing training data, incorrect tool use, premature termination due to output truncation, or lack of result verification. A key concept is the Harness — the runtime execution framework that determines how the model receives tasks, uses tools, manages context, handles exceptions, and decides when to stop. Therefore, a research agent's ability to improve an AI system depends on both the model and the resources it can access, the components it can modify, and whether it can identify true bottlenecks from experimental feedback.

Existing work such as PostTrainBench gives agents H100 GPUs and internet access but no starter code, training data, or hyperparameter configurations, forcing agents to build training pipelines from scratch. RSIGym takes a different approach: provide stable, reusable infrastructure as services so agents can focus on research decisions while retaining freedom to modify data, training setups, and the Harness.

Everything as a Service (EaaS) Design

RSIGym's core design is Everything as a Service (EaaS) . Research agents run in lightweight CPU-only containers. Training, inference, evaluation, and other heavy tasks are executed by remote services that agents call via APIs and accompanying Skill documentation.

For example, an agent submits data and a training configuration to the training service, which returns training metrics and a checkpoint identifier. That checkpoint can then be invoked through the inference interface and passed to evaluation and analysis pipelines.

Figure 1: RSIGym separates the research loop from underlying compute services, managed by a unified authorization service for permissions and budgets.

Five Core Services

Training Service : Executes training jobs, currently implemented with Tinker for LoRA fine-tuning, supporting optional custom loss functions.

Inference Service : Calls base models and trained checkpoints via an OpenAI-compatible Chat Completions interface.

Rollout Service : Invokes frontier models to generate training data or judge candidate results.

Evaluation Service : Assesses a "model + Harness" combination, returning overall score, per-task results, execution traces, and logs.

Sandbox Service : Provides isolated compute environments for data generation, test execution, and other experimental operations.

This design enables parallelism: training and evaluation run asynchronously, returning handles immediately so agents can continue analyzing logs, constructing data, or exploring other directions concurrently. Evaluation tasks can also run in parallel within their own sandboxes. Separation of research containers from underlying services establishes clear resource and permission boundaries; infrastructure credentials are held by the platform, and agents use per-run platform keys, reducing direct infrastructure exposure. Interfaces reuse familiar protocols: inference and rollout use Chat Completions format, evaluation uses Harbor configuration, sandbox uses E2B SDK. Crucially, feedback is not just a single score — agents can inspect loss curves, failed tasks, and execution traces to decide whether to modify data, adjust training, or fix the Harness.

Permission, Budget, and Audit as Experimental Design

When agents autonomously call services and modify code, experimental comparability depends on rule enforcement. RSIGym binds each run's permissions and budget to a single platform key. Services check permissions and balance before execution, record costs after completion, and agents can query remaining budget to allocate resources across data generation, training, and evaluation. This enables three research settings on the same infrastructure: researchers can restrict allowed teacher models, expose specific hyperparameters, and toggle public network access — all expressed via permission configs without rebuilding environments per experiment. Service isolation reduces rule-bypass opportunities, but full control requires covering credentials, network, and billing paths. The paper records an attempt to circumvent the platform budget by calling models directly; monitoring detected it, the run was terminated, and its results excluded. This illustrates that for RSI, permission control, budget accounting, and process auditing are themselves part of the experimental design.

RSI-Index: Measuring Joint Improvement of Model and Harness

With an experimental environment in place, a metric is needed to compare research outcomes. RSI-Index measures how much a research agent can improve a target system from a common starting point under a given budget.

In the main experiment, six frontier research agents each started from Qwen3.5-35B-A3B-Base with a minimal Harness , with both weights and Harness open for joint optimization. Each agent conducted independent research runs per benchmark, submitting the final system selected for that run.

Five benchmarks cover software engineering, terminal interaction, mathematics, scientific reasoning, and skill tasks: SWE-bench Verified, Terminal-Bench 2.0, AIME 2024/2025, GPQA Diamond, and SkillsBench (science, office, finance subsets). Several tasks require actual tool use and environment interaction. Each benchmark score uses avg@3 (three attempts per task, average reward across all attempts).

Per-benchmark normalized gain = (final_score − initial_score) ÷ (1 − initial_score) RSI-Index = mean of the five normalized gains

Example: a task improving from 20% to 60% fills 40 of the 80 remaining percentage points, yielding a normalized gain of 0.5. The main experiment allocated $500 platform service budget per benchmark per run ; agents' own inference costs and final official evaluation were billed separately. Each configuration was run once, and agents had access to final evaluation tasks during development, so scores reflect improvement on given tasks, not generalization to unseen tasks.

Six Research Agents Show Positive Gains but Different Strengths

In the Joint main experiment, all six agents achieved positive aggregate gains. Across 30 agent×benchmark combinations, 29 improved over the initial system.

Opus 5 achieved the highest RSI-Index: 0.4809 .

Ranking: Opus 5.5, DeepSeek V4.1 Flash, GPT-6 Astra, GPT-6.1 Sol, GPT-6 Sol.

Figure 2: Joint setting scores and RSI-Index. Each agent ran independently per benchmark; blue marks column best, red marks below initial system. Spend and Time are sums across five runs, including final official evaluation.

Aggregate ranking does not capture all differences. DeepSeek scored highest on SWE-bench Verified (53.33%); Opus 5.5 led on GPQA Diamond (85.33%); GPT-6 Sol, despite lowest aggregate index, topped SkillsBench (22.14%). This indicates clear task-specific variation in research capability. Improvement difficulty also varies widely: Opus 5 on AIME lifted score from 31.67% to 97.78%, filling ~96.75% of remaining space, while on Terminal-Bench and SkillsBench all agents filled less than a quarter of the gap.

Research trajectories reveal concrete phenomena beyond final rankings. Multiple agents fixed similar execution issues: checking edits before commit, preserving shell state, recovering from truncated output, detecting invalid loops, and verifying artifacts before task completion — changes that help models use existing capabilities more fully. Training also yields gains: in DeepSeek's SWE experiment, with the same Harness, the trained model's dev-set score rose from 0.344 to 0.600. However, more training does not reliably improve results: across ten same-Harness, same-task-set candidate comparisons, the stronger training update performed worse in eight cases. These small-scale comparisons provide directional evidence only. Budget increases from $500 to $1,000 improved three of four comparisons but degraded one; single-run results are insufficient to establish a stable budget-performance relationship. These observations make the "systems engineering" judgment concrete: data quality, weight updates, execution framework, experimental diagnosis, and candidate selection all affect final improvement.

Figure 3: Each group uses same Harness and task set, comparing pre/post training or lighter/stronger updates. X-axis shows percentage-point change relative to reference candidate; stars mark submitted checkpoints. Notes: a) reconstructed initial answer voting on synthetic task set; b) two attempts on divisor task; c) restored result after candidate selection.

Beyond the Main Benchmark: Harness-Only, Self-Training, and No-Internet Experiments

RSIGym supports other RSI experiment forms by adjusting initial system and permissions.

1. Improving an Existing Harness (DSH-harness)

Opus 5 modified DSH-harness while keeping Qwen3.6-35B-A3B-Instruct weights fixed. From execution logs, the agent identified issues: the framework could stop after output truncation, accept task completion without verification, or terminate background services still needed. After fixes, on the same 89 Terminal-Bench tasks, official avg@3 score rose from 30.34% to 40.82% (~10.5 percentage points), and tasks with three successful attempts increased from 15 to 26.

2. Model Training Itself (Qwen3.8-27B as both Agent and Target)

The team let Qwen3.8-27B act as both research agent and model to be improved. It performed data construction, training, candidate comparison, and submission, showing gains on six SkillsBench tasks in small-scale testing. However, on the full 37-task official evaluation, the final system scored 0.5626, slightly below the initial 0.5666. This demonstrates that autonomously completing the research pipeline and achieving reliable overall improvement are two distinct challenges.

3. No Public Network Access (Terminal-Bench Data Experiment)

In a Terminal-Bench data experiment, public network access was disabled, leaving only essential platform services and package mirrors. The agent generated ~2,600 trajectories, trained nine checkpoints, and passed the paper's audit, yet the score remained below the initial system. Trajectory analysis revealed that mixing reasoning content into regular messages in the data format hurt training effectiveness. Preserving a separate reasoning channel partially recovered performance but still did not surpass the baseline. These experiments surface both viable improvement paths and failed research processes, enabling further analysis of where problems arise.

Figure 4: Left: no-internet Data experiment; middle: fixed-weight DSH-harness improvement; right: Qwen3.8-27B self-improvement. Middle chart's connected success-count categories do not represent per-task flow; left dev eval vs official baseline and right small-scale vs full eval use different criteria, not directly comparable.

RSIGym's current experiments measure single-budget-constrained improvement of a target system. Whether improved systems can further enhance subsequent research capability, enabling sustained recursive progress, remains to be verified. For the research community, the open environment and experimental records allow more people to explore data strategies, training methods, Harness designs, and their joint optimization on shared infrastructure. RSI is systems engineering, not just a model problem. RSIGym aims to provide an environment where such systemic improvements can be autonomously explored, executed, and measured.