15 Agents Form an R&D Squad: End-to-End Multi-Agent Collaboration for Software Delivery

AI coding agents have advanced rapidly, yet end-to-end delivery time remains long because coding is only a small fraction of the process. The authors organize 15 agents into an R&D squad using AgentCore, covering roles such as requirement analysis, architecture exploration, implementation, testing, review, and deployment. A Leader agent schedules Workers, each with distinct Skills, Tools, and Context. Context passes forward: test failures return to the code session; architectural issues trigger re-research; repeated failures invoke a Human Gate for expert judgment.

Why Delivery Remains Slow Despite Faster Coding

A typical complex feature takes ~20 working days from requirement to release, with only 2 days for coding. Even a 10× coding speedup reduces the cycle to only 18.2 days. The remaining time is spent on requirement clarification, spec confirmation, cross-module integration, testing, review, and waiting. The goal is to let agents participate in these stages so context flows forward, verification drives the next step, and humans focus on goals, architecture, and risk.

AgentCore: The Execution Layer

AgentCore runs the squad. The 15 agents form a capability pool; the Leader decides task order, assignment, verification, and fallback. Task state is preserved — what was done, why, and what verification revealed — so subsequent agents can continue. Input can start from an Issue or Chat; output includes code, PR, test reports, review evidence, progress, blockers, and full execution trajectories.

Four Persistent Problems in Real-World Operation

Architecture constraints not conveyed: Agents may add new APIs instead of extending existing ones unless constraints are explicitly provided and continuously updated from code, docs, and experience.

Unexpected inputs: Open-ended user queries (e.g., asking a food-delivery app for Python code) fall outside predefined paths; they must be discovered and handled at runtime.

Recurring failures: Without continuous experience accumulation, agents repeat the same mistakes as code, rules, and environments evolve.

Regression risk: Fixing one case may break another; stable benchmarks and regression verification are needed after each optimization.

AgentLoop: The Optimization Layer

AgentLoop records and analyzes how tasks are executed: model/tool calls, failures, rework count, and result compliance. Traces, Sessions, and Trajectories feed a Pipeline → Dataset → Evaluation → Experiment cycle, producing reproducible samples, comparable metrics, and repeatable experiments. Three optimization modes are used:

Manual (expert-led): Experts define good outcomes, metrics, and rubrics; bad cases are added to datasets for experimental comparison.

Semi-auto (system-suggested): System reports high token consumption, inefficient steps, or persistent tool errors; experts act on the report.

Auto (experience library): Following a Retrieve-Augment-Context (RAC) approach, the system extracts SOPs, effective execution paths, and anti-patterns from trajectories and dynamically injects them into agents for reuse.

These modes combine: experts set goals, system assists analysis, and the experience library accumulates continuously.

Offline Benchmarking and Online Experience Accumulation

Offline: 20 high-quality expert commits were selected from the codebase; corresponding requirements were generated to form a benchmark. The agent team completes these tasks, and results are compared against expert implementations for solution deviation, code quality, verification completeness, and role division. Adjustments are validated via repeated experiments and regression tests.

Online: Real tasks generate new trajectories. The experience library distills SOPs, better execution paths, and tool-call anti-patterns (e.g., frequently failing call patterns) for future avoidance, and reusable solution paths for similar problems.

Multi-Agent Collaboration Loop

The squad iterates: Research → Explore → Implement → Verify → Compare. If acceptance criteria are not met, feedback triggers another iteration; otherwise, delivery proceeds. This mirrors the ReAct observe-act-feedback loop but across multiple roles. Two key practices: separate implementation and verification agents for independent perspective, and retain necessary context across iterations so each round builds on prior evidence and attempts.

Results in a Constrained Scenario: Trace to Trajectory

One production scenario is the Trace to Trajectory service, which converts trace data into trajectory data and encounters new parsing exceptions continuously. Unparsable cases are logged, alerts create Issues, and the squad handles localization, fix, and verification. In final validation on 19 test cases, ~10 achieved near-equivalent implementations to human experts, and 5 reached close-to-expert level. Delivery cycle dropped from ~1 month to under 1 week.

The core value is connecting requirements, implementation, verification, and feedback into a complete loop. AgentCore enables multi-agent teamwork; AgentLoop lets the team remember past work, accumulate experience from every delivery, and progressively improve subsequent tasks.