From Simple Q&A to Fact‑Checked Evidence: Building a Content Digital Employee at Gaode

Why a Content Digital Employee?

When an alert appears, developers must jump between monitoring, logs, code, and configuration, spending most of the time gathering context rather than diagnosing. The same difficulty exists for business‑level complaints, which start as natural‑language statements that need to be turned into factual snapshots before any investigation can begin.

From Coding Agents to Full‑Lifecycle Agents

Coding agents work well when the target repository, branch, and files are known, and success is measured by passing tests. A content digital employee, however, may start with only an alert card or a vague user statement, lacking any predefined entry point. Success criteria therefore shift to evidence completeness, candidate root‑cause credibility, and actionable suggestions.

Designing the Agent’s Working Environment

The team concluded that prompt engineering alone is insufficient; the reliability must be embedded in the surrounding environment. Key elements include state recording, explicit failure signals from tools, and a recoverable evidence index.

Four Pillars of Agent “Onboarding”

Job Boundary : Define which tasks the agent may handle, deliverables, stop‑conditions when evidence is insufficient, and actions that require human sign‑off.

Business Map & Knowledge : Provide an organizational view, domain semantics, and links to code, configuration, and runtime evidence.

Workspace & Permissions : Offer read‑only default access, minimal privileges, and require human confirmation for write operations.

Mentoring & Evaluation : Start with historical cases and read‑only tasks, use failure cases to calibrate routing and stopping rules, then gradually expand validated task coverage.

Constructing a Minimal Viable Business Map

The map must contain five categories of information: business domain, capability & responsibility boundaries, entry indexes, relationship edges (calls, dependencies, data flows), and evidence entry points. Rather than a static topology, the map should be navigable—able to locate a trustworthy first hop and suggest subsequent verification directions.

Skills and LLM Wiki

Skills encode repeatable investigation sequences, stop conditions, and judgment boundaries without hard‑coding volatile class or field names. The LLM Wiki stores curated, cross‑referenced knowledge pages that preserve source, update time, confidence, and conflict notes, acting as a semantic companion to the business map.

Evidence‑Chain Workflow (Root‑Cause Analysis)

Anchor the alert: extract application, time, metric, and rule information.

Validate rules and configuration: confirm thresholds, trigger conditions, and dimensions.

Analyze trends and distributions: distinguish spikes from sustained anomalies and identify affected services.

Cross‑verify logs and code: ensure logs and code agree on the failure path and downstream impact.

Correlate changes: check whether recent releases or config updates precede the anomaly and whether rollback restores normalcy.

Each step includes explicit stop‑conditions (e.g., insufficient evidence, transient spikes) and produces a structured output containing anomaly type, timeline, candidate causes, supporting evidence, counter‑evidence, and recommended actions.

Managing Context Windows

Because a single query can return thousands of lines of code or logs, the system no longer applies a uniform 8 KiB truncation. Instead, it allocates separate budgets per evidence type and returns navigable indexes (file, class, method, line numbers for code; timestamps and IDs for logs) so the agent can decide what to read in detail.

Compressing Evidence into Indexes

When evidence exceeds the context window, the system stores four items: a concise summary, a structural preview, a pointer to the original fragment, and a read‑back hint that tells the agent exactly where to fetch the next piece.

Evaluation and Quality Assurance

Evaluation looks beyond final text to verify that the agent met its objectives, collected complete evidence, respected permission boundaries, and kept latency and cost under control. Separate metrics assess the business map’s first‑hop accuracy, the LLM Wiki’s traceability, and the overall task success rate.

Future Roadmap

The next steps are to (1) solidify the business map for core domains, (2) seed high‑value topics into the LLM Wiki, (3) integrate three‑path querying (map → wiki → code), and (4) close the feedback loop by turning real‑world failures into regression tests and knowledge updates.