Reproducing SIGCOMM 2026 Network Systems in Two Hours with the Multi‑Agent RepLLM Framework

Background

Reproducing network‑system papers is often harder than understanding them because papers usually omit detailed module interfaces, data formats, environment setup, and scripts. A statistical analysis of SIGCOMM and NSDI papers from 2016‑2025 shows only ~38% provide open‑source code, and each paper typically needs to re‑implement 2.29 existing systems as baselines, making the cost of reproducibility very high.

Why Simple LLM Generation Fails

Directly feeding an entire paper to a large language model (LLM) does not work well: long papers cause key information loss, multi‑file projects lead to interface conflicts, and generated code may run without faithfully implementing the paper’s algorithms.

RepLLM Solution

RepLLM proposes an "AI reproduction team" of four specialized agents that share a common memory and work sequentially through four stages:

Content Parsing Agent : extracts sections, figures, formulas, and algorithms, building a structured knowledge tree with explicit identifiers and citation links.

Architecture Design Agent : converts high‑level paper descriptions into a system blueprint, defining modules, execution order, I/O interfaces, data structures, constraints, and environment dependencies, thereby avoiding interface drift and circular dependencies.

Code Generation Agent : uses a structured chain‑of‑thought (SCoT) to translate the algorithmic description into an intermediate representation with typed inputs/outputs, conditionals, loops, and data dependencies, then emits concrete code without committing to a single language early on.

Audit & Repair Agent : runs static checks for syntax, interface, and dependency issues, then executes the generated system in an isolated sandbox, applying dynamic patches for port conflicts, missing dependencies, data‑format errors, or orchestration failures.

The shared memory stores the paper content, system architecture, code state, and repair logs, ensuring all agents operate on the latest unified context. Ablation studies show that removing shared memory or interface memory leads to empty‑shell programs or broken module interactions.

Evaluation on Seven Representative Network Systems

RepLLM was tested on seven diverse systems—NCFlow, GRooT, Rateless IBLT, SelfStarter, NetDice, Teal, and NetChain—covering traffic engineering, DNS validation, coding theory, configuration mining, probabilistic verification, and programmable data planes. Implementations span Python, C++, Go, and P4, and depend on tools such as Gurobi, CUDA, Mininet, and BMv2.

Results:

All generated code compiled, ran, and successfully loaded the original datasets.

Compared with single‑call LLM generation (e.g., Claude Code), RepLLM achieved higher runnable‑code rates and more faithful integration of real data‑plane and control‑plane components, especially for NetChain.

Automatic generation took 38–84 minutes per task; manual calibration added 23–93 minutes, yielding total reproduction times of 82–138 minutes, roughly two hours.

Post‑calibration, the reproduced systems matched original results on key metrics (e.g., NCFlow throughput, NetDice test suite, GRooT violation detection, Teal’s FlowGNN/COMA/ADMM pipelines, SelfStarter ACL detection, Rateless IBLT protocol verification, NetChain’s P4 data‑plane and Python control‑plane coordination).

Overall, RepLLM reproduced about 95% of the original benchmark results while reducing token usage and context passing.

Implications and Future Work

Beyond speed, RepLLM aims to close the research automation loop: paper understanding → system design → code implementation → environment construction → execution verification → result reproduction. It can help researchers quickly build baselines, verify ideas, and lower cross‑domain reproduction costs, while also serving authors and reviewers as a reproducibility‑checking aid.

Current limitations include difficulty handling kernel/driver modifications, specialized hardware, or large‑scale cluster dependencies that cannot be fully validated in lightweight sandboxes. Future work will extend support for Kubernetes, LLVM, and other large‑scale projects, and further refine the agent toolkit for broader scientific tasks such as literature review, data analysis, and manuscript drafting.