Design Review of a Multi‑Agent Zero‑Ops System: 4 Agents, 7 Skills, and 9 Engineering Judgments

Recent articles have clarified how multi‑agent systems should look, but few discuss the practical engineering challenges of turning a demo into a production‑ready, zero‑human‑intervention workflow. The GOAI World AI Open Competition’s Agent Infra track posed the concrete problem: move a multi‑agent system from demo to production.

Demo Overview

OpsPilot Zero is a deliberately small demo that automates a typical e‑commerce incident where a customer complaint at 10:15 triggers a cascade of alerts, logs, traces, and slow‑SQL queries. The goal is to shrink the end‑to‑end response time from the usual 30‑60 minutes to 5‑10 minutes, with zero human involvement for low‑risk actions and an approval workflow for higher‑risk steps.

Agent Responsibilities

Alert Intake Agent : aggregates scattered signals into a single incident object.

RCA Analyst Agent : correlates logs, traces, configuration changes, and slow queries, ranks root‑cause candidates, and produces an evidence chain.

Remediation Planner Agent : generates a ranked action list, rollback points, and approval requirements.

Recovery Verifier Agent : executes allowed actions, validates results with probes, and produces a final recovery report.

Skill Layer

Seven skills are defined (e.g., alert-fusion, impact-mapping, log-trace-rca, data-advisor, remediation-plan, risk-guard, recovery-verify), each exposing a clear input/output contract that can be independently tested.

9 Engineering Judgments

Guardrails in Skills : every skill must include a guardrail segment that defines boundaries and refusal conditions; for example, the log-trace-rca skill forbids a root‑cause conclusion when only a single piece of evidence is available.

At Least Two Independent Evidences : a root‑cause conclusion must cite two independent sources (e.g., logs + trace) and record confidence scores.

Four‑Level Risk Grading (L0‑L3) : actions are classified from read‑only diagnostics (L0) to high‑risk, non‑executed changes (L3). Only L0 and L1 actions are auto‑executed; L2/L3 require manual approval.

Unified Mock‑Production Schema : mock data sources and real data sources must share identical input/output schemas so that swapping in production data requires no code changes.

Registry‑Based Prompt/Skill/AgentSpec : inline definitions are acceptable for a demo, but production must store prompts, skills, and agent specifications in a registry (e.g., Nacos) to support versioning, gray‑release, and audit.

Explicit Missing‑Evidence Section : reports must contain an “Evidence Gap” paragraph that lists unverified hypotheses and suggests telemetry to collect.

Separate Engineering for Single‑Run vs. Repeated Runs : a production‑ready system needs a golden‑case and bad‑case dataset, stability checks (consistent output over ten runs), and idempotent action keys.

Rollbackability Over Automation Rate : the most important metric is the ability to roll back; a 90 % automated system without rollback points can cause catastrophic failures.

Transparency of Limitations : the demo openly documents its constraints (mock data, inline specs, no evaluation set, single‑cloud deployment) to set realistic expectations.

Demo Execution and Report

Running the mock scenario where the DB connection pool is exhausted produces a concise incident report. The report includes an incident timeline, a root‑cause section ( db_pool_exhausted with confidence 0.91 and three independent evidences), a remediation plan (rollback configuration, risk level L1, auto‑execute flag “yes”), execution and verification results, and telemetry recommendations that highlight missing data.

Limitations and Path to Production

All data are mock; real logs, traces, and slow‑SQL are noisier.

Agent specifications are inline, lacking a registry for version control.

No evaluation set; only two mock scenarios are covered.

No trace dashboard; observability relies on raw JSON logs.

Only tested in a single cloud environment.

Addressing these five gaps is essential for moving from a minimal closed‑loop demo to a production‑grade system.

Competition Context

OpsPilot Zero was built for the GOAI World AI Open Competition – Agent Infra track, which rewards systems that demonstrate multi‑agent collaboration, skill reusability, robust engineering, and open‑source potential. The track emphasizes real‑world scenarios, autonomous closed‑loops, and engineering rigor over raw AI cleverness.

For readers interested in the detailed implementation, the author invites them to join the DingTalk group (QR code shown below) for code access, office‑hour discussions, and further engineering deep‑dives.