Can You Trust AI to Code a Million‑Line Backend System? Lessons from Tencent’s LEGO Harness Engineering

Introduction

When the spotlight on AI coding focuses on front‑end tasks like generating pages or apps, a critical question is often ignored: Can AI write code for a backend system where a single mistake could cause a nationwide outage?

The Tencent CDN LEGO project is such a system. It contains over 1 million lines of core code and more than 3 million lines of deeply modified third‑party libraries , serving billions of requests daily. The system must handle uncontrolled clients, unpredictable origins, multiple protocols, countless configurations, and a full‑scale public‑facing attack surface. The combination of these dimensions leads to a theoretical configuration space of 13,824 × N possibilities, meaning a single erroneous line could trigger a global incident.

Background and Challenges

LEGO is the core access layer of Tencent CDN, responsible for traffic scheduling, protocol parsing, security protection, and cache acceleration. Its complexity stems from:

Code scale: >1 M lines, asynchronous non‑blocking design requiring deep expertise in concurrency.

Third‑party dependencies: >3 M lines of modified libraries (OpenSSL, QUIC, Lua, JavaScript, etc.).

Service scale: trillions of requests per day; any performance, stability, or security issue can be amplified.

Additional challenges include:

Numerous uncontrolled factors (clients, origins, protocols, configurations).

Complex asynchronous semantics (long Future/Promise chains, lambda lifetimes, thread‑safe state synchronization).

Zero tolerance for failure (first‑hop, stateless, streaming processing).

Strict protocol security requirements (HTTP RFC compliance, cache safety, injection protection).

Combinatorial dimension explosion (multiple request protocols, back‑end protocols, TLS versions, cache states, domain configs, script logic, security rules).

Industry Landscape and Nonstop Project

AI coding is already appearing in industry case studies, but its suitability for ultra‑large, highly uncertain backend systems remains unproven. To explore the limits, the team built a Rust‑based Nonstop proxy framework in 20 days with zero human code, using it as a testbed for AI‑generated code.

Nonstop features:

Full L4/L7 proxy support.

HTTP/3 and QUIC protocols.

Built‑in WAF for deep defense.

V8 JavaScript engine for edge computing.

Single‑binary deployment with hot‑load zero‑downtime.

In 20 days, a single developer plus AI delivered a system handling 42,052 QPS with 5,000 concurrent connections, 0 errors, P50 latency of 1.1 ms, and six layers of security.

Why AI Coding Fails in Large Projects

From 57 real cases, the team identified 13 problem types and five root causes. The most critical issue is AI’s inability to say “I don’t know,” leading to confident but wrong outputs. Other problems include hallucinations (fabricated APIs or RFC sections), incomplete changes, pattern‑matching instead of verification, and lack of environment awareness.

Harness Engineering Architecture

The solution is not to “use AI” but to “harness AI.” The team designed a five‑layer architecture centered on three pillars: context, constraints, and feedback . The layers form a closed loop that ensures AI‑generated code passes through rigorous checks before deployment.

1. Core Idea

Restrict AI to a single module, file, or function, providing explicit context, strict constraints, and continuous feedback.

2. Five‑Layer Design

Each layer adds a guard:

Layer 1: Permission‑based security foundation.

Layer 2: Code rules acting as a compiler.

Layer 3: Process constraints—tests cannot be skipped (implementation → unit test → code review).

Task dependencies are expressed as a DAG (e.g., implementation blocks testing, testing blocks review). This guarantees that no step proceeds without the previous one passing.

3. Constraints

Concrete constraints derived from real incidents include:

Single‑project research: only one competitor is investigated at a time.

Network‑free operation: AI must not perform live network calls.

Local‑only analysis: if source code is missing, skip the step.

No modification of lego_server code.

Strict search scope to avoid contaminating system directories.

These constraints turn vague expectations (e.g., “write high‑quality code”) into enforceable rules (e.g., “prohibit naked new, require unique_ptr ”).

4. Multi‑Model Adversarial Code Review (CR)

Single‑model CR suffers from three blind spots: knowledge gaps, attention drift on large diffs, and confirmation bias. The team runs three independent models in parallel, aggregates their findings, and performs cross‑validation:

If two models flag the same issue, confidence is high.

Unique findings trigger additional verification rounds.

The process includes debate‑style discussions (agree/disagree/maintain) and automatic convergence when no new issues appear.

5. Feedback Loops

Three feedback channels accelerate learning:

Automatic hook collection of runtime data.

Pitfall journal logging of AI failures.

Inline .md feedback attached to generated artifacts.

Each pitfall is turned into a rule, validated via A/B experiments, and added to the shared Skill repository.

Practical Cases

CPU‑info read/write race fix : AI identified three possible solutions (ReadWriteLock, atomic<shared_ptr>, double‑buffer with atomic index), generated test cases, and the team chose a zero‑overhead solution, reducing development time from 5 days to 1 day.

Efficiency gains : Across the project, AI‑assisted competitor research, design, protocol security testing, and code review saw speed‑ups of 2‑4×, and overall engineering efficiency improved by roughly 20 % after accounting for learning curves.

Differentiation and Challenges

Compared with typical industry practices (single‑model CR, static scans, fixed rounds), LEGO’s approach offers:

Multi‑model parallel review with cross‑validation.

Systematic knowledge base built from 57 real issues (34 problem types, 5 root causes).

Three‑time‑scale feedback (real‑time hooks, daily logs, permanent rules).

Remaining challenges include a 36 % false‑positive rate in AI‑detected issues, documentation explosion (many generated files), AI over‑confidence reducing human review willingness, and the risk of skill degradation among engineers.

AI Coding Era – Role Evolution and Team Building

Backend engineers must shift from pure coders to AI operators , designing constraints and validating AI output. Senior engineers become Harness Engineers , architects focus on human‑AI collaboration, and test/security engineers evolve into AI quality and safety specialists.

The transformation follows three phases:

Months 1‑2 (Learn to use): Adopt end‑to‑end AI workflow, adversarial CR, and security rules.

Months 2‑4 (Learn to build): Core members author team‑specific Skills, validate via A/B experiments, and share knowledge.

Months 4‑12 (Learn to evolve): Automate Harness pipelines, enable cross‑team knowledge sharing, and continuously monitor AI impact.

The team must balance three attitudes: caution (review every AI line), aggression (deeply apply AI in high‑frequency scenarios), and embrace (promote AI as a cultural asset).

Conclusion

The engineering system—not the model or prompt—is the lasting asset. AI coding is not about replacing developers; it is about redefining collaboration. LEGO’s Harness Engineering turns every pitfall into a rule, every rule into a reusable Skill, and every Skill into a knowledge multiplier, creating a sustainable, self‑evolving development ecosystem.