From Chat to Full SDLC: 7 Months Building Trustworthy AI Agents for a PC Team

Overview

This article documents a PC development team's seven-month experience deploying AI agents (called "digital partners") as full-fledged team members. Starting from ad-hoc queries in a Feishu group, the team progressed to multi-agent autonomous collaboration across the entire software development lifecycle (SDLC), including requirements analysis, development, testing, and release. The report covers organizational governance, technical infrastructure, collaboration protocols, memory/skill systems, and three end-to-end case studies.

1. From Model to Working R&D Agent

A language model alone cannot execute work. The team built an execution environment where agents can read repositories, run commands, query logs, and call business APIs within authorized scopes. The loop of model, tools, execution, and persistent records enables real task completion. A key early example: a tester asked for a movie playback URL. The agent initially refused (no CMS access), but after the tester challenged it to find existing internal search capabilities, the agent traced code call chains, located a reusable search interface, verified it, and within five minutes delivered a reusable Python script, a registered skill, and a Wiki entry. Two collaboration etiquette fixes emerged: using proper @-mentions for notifications and actually attaching referenced files.

2. Identity Governance: Five-Layer Architecture

The team established a principle: "Without an employee identity, it's a tool; with an employee identity, it's a digital employee." This aligns with industry trends (Gartner 2026 IAM for AI agents, IETF OAuth extensions, OWASP Agentic Applications Top 10). They designed a five-layer governance stack:

Layer 1: Identity

Responsibility: Digital employee roster & approval entry

Key Mechanisms: ID, role, manager, lifecycle registration

Layer 2: Authentication

Responsibility: Workload identity & explicit delegation

Key Mechanisms: Token exchange, delegation credentials, expiry

Layer 3: Authorization

Responsibility: Execution permission approval & separation of duties

Key Mechanisms: Intersection not union, expiry reclamation, high-risk human review

Layer 4: Audit/Accountability

Responsibility: Anchor each digital employee to a human Owner

Key Mechanisms: Operation traces, responsibility tracing, audit replay

Layer 5: Collaboration

Responsibility: IM/OA membership & inter-agent authorization

Key Mechanisms: OA account is the result; must build bottom-up from L1-L4

Seven design principles include: permissions requested independently (no inheritance), intersection-effective, expiry reclamation, high-risk always human-reviewed, guardrails at authorization layer (not prompt layer). The "designated executor" uses RFC 8693 token exchange (sub + act) for explicit, time-bound delegation.

3. Five-Stage Evolution

Stage 1: Assistant

Key Change: Info retrieval → content generation

PC Practice: Human drives context & execution

Stage 2: Agent

Key Change: Answer → act

PC Practice: Zcode, Trae read code, call tools, modify, verify

Stage 3: Worker

Key Change: One-off → long-term role

PC Practice: cc-connect gives Feishu entry, project identity, workspace, persistent sessions, scheduling, result callback

Stage 4: Manager/Worker

Key Change: Individual → collaborative orchestration

PC Practice: Manager receives tasks, routes by capability, tracks status, evidence

Stage 5: Digital Employee

Key Change: Technical orchestration → organizational responsibility

PC Practice: Define role duties, service targets, authorization, evaluation, learning gates, human Owner

The growth loop: Task → Action → Result → Human/System Feedback → Validated Memory → Reusable Skill → Better Action. Workspace isolates project context; Memory stores validated experience; Skill turns success into repeatable capability; Workflow combines skills for predictable delivery.

4. Collaboration Layers: cc-connect & IM-Native

4.1 cc-connect: Bridging IM and Coding Runtimes

Three interfaces separate concerns: IM layer (message send/receive), Adapter (session start/resume), Session object (running execution). This allows multiple coding agents (GLM, Kimi, etc.) behind a stable Feishu entry. Message IDs prevent attachment confusion. Persistent sessions keyed by "group + topic root message" provide seven months of unbroken history.

4.2 IM-Native: Task Semantics in Chat

Chat must carry both human-readable conversation and machine-verifiable collaboration semantics. State stored in MongoDB (execution records), Feishu messages as entry/audit, dashboards for business status, Redis for acceleration. Key identifiers prevent confusion:

conversation_key : Tenant, platform, group, stable topic root — prevents cross-group/topic context pollution

run_id / task_id : One collaboration goal / logical task — prevents mixing multiple deliverables in one chat

attempt_id : One execution attempt per task — prevents old retry results overwriting new

revision : Substantive requirement/artifact modification round — prevents treating clarification as rework or missing real changes

action_id : Single replayable action — prevents duplicate notifications, dispatches, side-effects on restart

business_version / storage_version : Approved content / storage concurrency — prevents heartbeat writes invalidating approvals, stale writes

Task states: pending, dispatched, running, delivered, accepted, waiting_input, failed, cancelled, uncertain (external result unknown). Outbox pattern ensures notification durability: write pending notification to persistent log, sync, deliver, mark completed; on restart replay pending only. Four automated tests cover in-process dispatch, multi-worker aggregation, display/execution separation, and dispatch notification across crash window.

5. Protocol Correctness vs. Engineering Completeness

IM-Native v3 defines eight collaboration modes (routing, expert consultation, round-table, parallel, voting, relay, production review, red-blue). Every message records three identities: initiator (real human), sender (current event sender), run_as (executing agent). 34 acceptance criteria; local deterministic verification passed 34/34, but integration testing with dedicated test identities/groups waited three weeks. Example: round-table convergence requires ≥2 rounds, then either 90s silence or Captain explicit DONE/👍; late messages don't trigger re-summary. Worker offline handled via short-lease heartbeat directory; deterministic reassignment to protocol-compatible standby; late original results suppressed.

Live drill: Captain agent coordinated "PC playback stutter auto-attribution" solution. Log diagnosis, source mapping, user feedback experts worked in parallel; adversarial reviewer challenged all. Completed quickly with multiple signal-to-root-cause mappings, conflict resolutions, and blind-spot identification. One ruling: log expert blamed rendering layer; source expert identified playback kernel detection threshold and scheduling chain; adversarial reviewer upheld latter and provided routing criteria.

6. Organizational Adaptation: Roles, Handoffs, Complexity

Digital partners join existing org structures (Product, Dev, QA, Release, Captain). Coasean view: coordination costs remain. Explicit input/output contracts per role:

Role Contracts

Product/Req : Inputs — Business goal, current state, affected users, scope. Deliverable — Requirement spec, acceptance criteria, out-of-scope, human confirmations

Development : Inputs — Confirmed req, repo & version, implementation constraints. Deliverable — Merge request, commits, change explanation, self-test evidence

QA : Inputs — Complete req, deployed env & version. Deliverable — Coverage matrix, pass/fail cases, uncovered items, risks

Release : Inputs — Approved artifacts, test conclusions, approvals, release window. Deliverable — Pipeline records, production version, rollback basis, release receipt

Captain : Inputs — All role artifacts & authorized decisions. Deliverable — Current status, next owner, blockers, traceable handoffs

Team size increases coupling (n(n-1)/2). Near-decomposability (Simon) suggests clear work units connected by requirement versions, interface contracts, test results, delivery artifacts. Role isolation, limited parallelism, human intervention thresholds keep complexity inspectable. Shared attention constraints: group messages show conclusions, artifact links, decisions needing humans; derivations/logs/evidence go to expandable Wiki.

7. Memory, Skills, and Continuous Improvement

7.1 Documentation Must Preserve Correction Process

Post-task summaries retain the pivot points: why initial search scope was wrong, which human correction changed direction, what evidence overturned judgments. Three summary densities: full audit, key process (retains pivots, corrections, decisions, evidence), minimal (problem + conclusion). Auto-routing selects density by reader intent.

7.2 QA Loop: Chat ≠ Organizational Memory

Hard rule: every task conclusion must be written to Feishu Wiki (quarter/week/topic hierarchy, same page updated per round). Six fixed sections: user request, execution steps, key evidence, actual changes, verification results, residual risks. No "analysis only" excuses. Closed loop: question → dispatch → receipt → gate → archive, all in group. Evidence, environment, input version, authorizer required. Missing evidence = BLOCKED; mismatch = FAIL; only PASS on acceptance.

7.3 Short-Play Localization Correction Reused in PiP Task

Initial short-play search focused on player kernel repo; human pointed to PC WebApp. Agent switched to business frontend repo, used domain terms (series, short-play, video flow), located implementation. Formalized into flow-feature-locate skill: identify product entry from business semantics → locate functional module → confirm with code evidence. Updated repo index, skill entry, feedback memory. Later PiP task reused this across client, frontend, player, backend repos.

7.4 Retrieval, Versioning, Counterexamples Keep Memory Correctable

Transactive memory (Wegner): group needs to know "who knows what" and how to access. Internal memory library indexes sessions, Wiki, local memory. Reusable experience carries source path, content version, applicability conditions, feedback. Content hash + path for versioning; re-read current source when needed. Negative feedback pins to specific content version & task scope to prevent re-ranking. Whole-page Wiki not universally trusted; granular isolation. Memory evolution ledger: daily candidate extraction, promotion via replay validation, holdout, canary, human approval (different person). Corrected memories auto-quarantined. Ledger stores sanitized summaries + fingerprints, not raw chats.

Skill accumulation from real incidents: desktop "episode list not showing" regression caused by missing parentheses changing boolean precedence. Postmortem added code review check items, test combinations, pipeline gate candidates. Reliability measured by whether errors become mechanisms that make recurrence harder.

8. SDLC Trial: Three Real Requirements

Aligned with Anthropic's AI-native SDLC Playbook (Plan, Design, Build, Test, Deploy, Maintain). Team's "PC-AI-Native SDLC Test Group" uses frontend phases: Plan, UI, FeDev, FeTest, FeRelease, Observe, Close. Backend needs adapted routing. Handoff must answer: which version, evidence location, what next role checks. MR, commit, build version, pipeline run_id, approval ticket — each has distinct purpose; "done" text cannot substitute.

Five human gates retained:

SDLC Sponsor : Role resource conflicts, over-authorization — Intervention: Coordination failure or beyond authority

Product Owner : Goal, scope, acceptance semantics — Intervention: Requirement definition & final acceptance

Code Owner : High-risk change approval — Intervention: Changes touching high-risk list

QA Owner : Exception release — Intervention: Test not fully covered but release requested

Release Manager : First production release & rollback — Intervention: Release/rollback nodes

Digital partners follow role instructions: discussion participation ≠ approval authority.

8.1 Requirement A: One Copy Change Across Three Services

Change "next episode" hint to "Contains AI-generated content". Product found copy in portal service; dev discovered second hardcode in player service; scope check revealed third in channel service. Initiated one afternoon; agent created tickets; dev agent blocked pre-build on player service hardcode (prevented missed change). Three MRs ready, independent reviews passed same day. Scope refined through v0.4 (PCW/PCA, album/play flows, no click/style change, fallback copy unchanged, PPC player out of scope). Test initially 7/10 cases executed; QA challenged coverage. Final 21 full-chain cases, risk-accepted; pre-release retest passed two points. Code merged after explicit human authorization. Two services released; player service separate. Production verification: AI samples show new copy, non-target unchanged. Product Owner confirmed acceptance. Process exposed hidden org knowledge: which branch for test, who approves merge, when to notify QA, which uncovered items must be listed.

8.2 Requirement B: Homepage Nav Links to HTTPS

Dev produced MR, test package; QA independent 20 cases passed. Flow stopped at Product Owner request (cancelled). During test, QA lead said "looks like it can pass". Captain treated as gate pass, asked downstream to release. QA lead corrected: "You're misinterpreting me." Agent rolled back; only explicit "test pass" advanced. Revealed need for speaker identity, artifact binding, formal decision flag, context awareness. Subsequent issues: test executor misassignment, confirmer binding errors, duplicate @, long messages. All recorded as new conventions.

8.3 Requirement C: Privacy Policy Link to Specific HTTP URL

Pre-holiday: scope confirm, UI confirm, dev, 26 tests passed. Release whitelist blocked; holiday freeze stopped pipeline (zero production changes). Post-holiday: human challenge triggered re-check of temp branch assumptions. Actual package contained other routine dev changes; original release premise invalid. Lead ruled: abort original, switch to formal baseline. Dev re-cherry-picked two changes, new MRs, unit tests green, rebuild & retest. Two recovery details: (1) A/B adjudication: admin demanded code review before release; review showed no blockers but minor risks; delivery did "hypothesis falsification" — production not in assumed state; requested A (ship known increment) vs B (rebase formal baseline, rebuild, retest); authorized person chose B. Dispatch convention fixed: only "what requirements, which branch", no inferences. (2) Test environment drift: two cases failed due to daily build pollution; switched to isolated lane, rebuilt package. Hosts modification required test machine admin; agent blocked repeatedly, escalated to manual desktop. Filing site anomaly led to defect ticket with evidence. QA lead sent manual verification video; classified as test tool issue, not requirement defect. Gate closed: majority direct pass, two manual verified. Night approvals stalled; system reminded, escalated, then entered night silence — no third reminder, no auto-decision. Rebuild environment overwritten by manual old package; isolated lane, re-package, re-test. Lesson: test report must bind requirement, code, build, actual env version. Final: pipeline success, approval closed, production version matches scope; observation passed; roles confirmed closure. PMS fields still pending. Ten+ collaboration incidents archived; largest: temp vs formal baseline assumption error. Others: test lane pollution, hosts auth blocks, filing site false defect, PMS reporter mapping missing, defect linked wrong project. Process improvements landed: dispatch wording standard, PMS write-back fallback, test tool manual verification precedent, test exclusive lane practice.

9. Measurable Outcomes & Adaptation Timeline

Three trials show progression: cross-service localization → independent testing → long-cycle delivery state maintenance & production acceptance. Evaluation objects: code diffs, test coverage, env versions, approval artifacts, production receipts, human final confirmations. Not all requirements automatable; no universal efficiency multiplier. Human corrections, routing revisions, env interference remain. Architectural decisions (M0): process occurrence = SDLC service state transition success (not LLM judgment or chat); Captain = unified entry, no task runtime state; PMS = business requirements, SDLC service = work items & execution, chat = collaboration/notification/visibility only.

10. Efficiency Metrics & Adaptation Period

Delivery engineer time initially overcounted by including pipeline/build/deploy/smoke time. Corrected split: agent does trigger/verify/receipt; rest is system execution. Metrics to track long-term:

End-to-end cycle : Scope confirm → final acceptance; separate freeze & business wait — Where work stalls: execution, approval, queue

Human actual effort : Request, clarify, review, correct, takeover recorded separately — Whether automation truly reduces colleague burden

Agent action vs system wait : Timeline distinguishes tool ops, model processing, build, external wait — Optimize model, process, or infra

First-pass vs rework : By requirement version: first acceptance, rework cause, affected phase — Fast output causing more later fixes?

Delivery traceability : Requirement, commit, build, test env, approval, prod version align? — Result independently verifiable & recoverable

Collaboration noise & interruption : Misdispatch, duplicate notify, wrong ruling, human correction count — Team attention consumed by automation?

Quarterly ops reports show sustained production use: user input rounds, independent sessions, active days growing; tasks involve heavy real tool interactions; active workers cover multiple logical roles; Wiki evidence base exists but incomplete. Value report estimated hours saved via session sampling; must read with caveats: model estimate based on task classification & time assumptions, not measured KPI; token telemetry coverage incomplete.

Adaptation timeline: "able to converse" → "able to collaborate" ~3 months; "able to collaborate" → "dare to entrust" ongoing. Mid-year: agents just completed one decent business dialogue, cost prohibitive. Self-assessment scored low: "end-to-end delivery under strong human escort, not yet stable, low-human, measurable continuous operation." M0 loop closed soon after. Adaptation is bidirectional: humans learned to specify acceptance criteria (directly affects delivery quality), define authorization boundaries (vague auth blocked by gates), read evidence chains not conclusions. Agents learned to say "cannot confirm" when evidence insufficient. Tuckman's stages (forming, storming, norming, performing) analogize. Three simultaneous adaptation fronts: humans learning context provision & result verification; agent external memory/skills absorbing domain semantics & ops rules; collaboration system fixing message delivery, recovery, permissions, version binding. Teaching longer prompts doesn't solve the latter two.

Phased pilot design: Phase 1 high-frequency verifiable small tasks (info lookup, doc organization) — check citation & result reliability. Phase 2 repeat similar tasks — observe if last correction applies next time. Phase 3 integrate dev & independent test — require handoff artifacts & env versions verifiable. Phase 4 extend to release & long-term maintenance — humans decide at explicit gates, practice cancel/restart/failure recovery. Each promotion needs real task evidence. Allow asymmetric capability growth; keep fallback to human.

11. Seven Actionable Recommendations

Identity first, account later. Register digital employees in IAM roster, anchor responsible Owner, then discuss OA account. Skipping governance creates "badged but unmanaged" shadow employees.

Gates first, autonomy later. High-risk list always human-reviewed; autonomy expands with evidence maturity, not LLM self-judgment. Process occurrence = state system transition record.

Small closed loop first, full chain later. Run one real requirement end-to-end before platformizing. Honest low maturity self-assessment beats pretty dashboard numbers.

Teach agents to say "cannot confirm". Evidence culture > answers. Conclusions tagged: direct fact, code inference, indirect inference, cannot confirm. No guessing without evidence loop.

Conclusions must archive. Chat disappears; Wiki persists. Every round conclusion archived on the spot with request, evidence, changes, verification, residual risks. Missing evidence = BLOCKED, not forced answer.

Govern memory. Experience adoption goes Replay, Holdout, Canary, human approval (different proposer). Corrected memories auto-isolated. Ledger stores sanitized summaries + fingerprints, not raw chats.

Preserve failures. Failed paths, stop conditions, adjustment rationale are reusable. Honest postmortems of misjudgments help successors more than success lists.

12. Closing: The "Character" of Digital Partners

Trust comes from three mundane traits: Predictable — they operate within rules, night ops same as day, same constraints. Auditable — every step has evidence, archive, replay. Growable — experience becomes governed assets. Organizations evolved over centuries to solve "how large-group collaboration stays orderly", not "how to make people faster". Digital partners integrate into that evolution, not bypass it. Managing them — guard identity, guard gates, keep evidence, admit mistakes — is the core experience this article shares. The accumulated scripts, evidence, requirement conventions, debugging methods let the next colleague pick up on the same collaboration chain. That's when an R&D agent truly becomes an efficient, trustworthy digital partner.

References

Anthropic, The AI-Native SDLC Playbook, 2026-08-21; How Anthropic secures its AI-native software development lifecycle, 2026-07-21

Anthropic, Claude Tag Slack public beta announcement, 2026-06; Claude Code into Team/Enterprise plans, 2025-08; Claude Code Agent Teams official docs (experimental)

Gartner, 2026 Security Trends: IAM for AI Agents / From Tools to Actors

IETF, RFC 8693 OAuth 2.0 Token Exchange; AI Agent On-Behalf-Of authorization draft

OWASP, Agentic Applications Top 10 (2026), ASI03 Identity & Privilege Abuse

Okta 2025 Non-Human Identity Strategy Survey

Coase, The Nature of the Firm, 1937; Simon, The Architecture of Complexity, 1962; Tuckman, Developmental Sequence in Small Groups, 1965; Wegner, Transactive Memory

John Holland, Hidden Order: How Adaptation Builds Complexity; Peter Senge, The Fifth Discipline

Snyk, Public Agent Skills Security Research (13.4% critical issues)

https://github.com/chenhg5/cc-connect

Code example

同一人群、同一时间范围、同一有效播放定义下的恒等分解（模拟数据）
总时长 T = 有效播放设备数 UV × 每设备播放次数(VV / UV) × 每次播放时长(T / VV)
A：3000 / 600 = 5.00 次/设备
B：2200 / 615 ≈ 3.58 次/设备
次数减少，单次时长基本持平；总时长下降主要落在每设备播放次数这一项。