How AI Coding Powered a 150K‑Line Core System Refactor at Qunar

Introduction: AI Coding Beyond Small Tasks

AI coding has become common for clear, scoped tasks, but this article explores its use in a 150 k‑line, high‑concurrency core system refactor where any uncontrolled change could cause online risk. The authors aim to answer three questions: can AI move from assistance to driving large‑scale refactoring, how to keep the AI execution controllable, verifiable, and rollback‑able, and whether prompts, toolchains, or engineering constraints provide the reusable experience.

1. The Difficulty of Large‑Scale Refactoring: System‑Level Uncertainty

Unlike small tasks that focus on local functionality, large refactoring must ensure system‑wide behavior consistency, performance, stability, and safe rollback. Four types of uncertainty determine success:

Behavior uncertainty: request/response semantics must remain identical across edge cases.

Impact uncertainty: changes must not unintentionally affect unrelated flows or caches.

Verification uncertainty: manual tests alone cannot cover the complex traffic combinations in production.

Risk uncertainty: any online deviation must be quickly detected, isolated, and rolled back.

If these uncertainties are not explicitly handled, AI may appear to finish the code changes while the team cannot confirm that the system meets launch criteria.

2. Why Traditional Refactoring Is Costly

Traditional approaches require massive manual effort to map system structure, run extensive tests, and record analysis, leading to long cycles, incomplete system knowledge, insufficient test coverage, and fragmented documentation.

3. Turning Uncertainty into Engineering Constraints (Harness & Loop)

The methodology introduces two key concepts:

Harness: defines the target, boundaries, context, tools, and stop conditions, limiting what AI can modify.

Loop: provides a feedback cycle—plan, modify, verify, analyze, record, and decide the next step (continue, retry, or hand over to a human).

Both are required to keep AI from drifting in an open space and to ensure each execution is bounded and observable.

4. Four‑Stage Closed Loop for Refactoring

The work is split into planning, design, execution, and acceptance stages. Planning clarifies goals and metrics; design produces technical solutions and milestone breakdowns; execution runs AI‑driven tasks with verification and recording; acceptance aggregates test results, diffs, and performance data to decide whether to proceed or release.

5. Human‑AI Division of Labor

Humans remain responsible for goal definition, boundary judgment, architectural decisions, risk acceptance, and final release decisions. AI handles code scanning, assisted design, code modification, test execution, diff analysis, and documentation.

6. Task Definition and Documentation

Each Task is the smallest AI‑stable unit and must explicitly state:

What to do (objective, business background, deliverables).

Where to change (affected modules, code list, dependencies).

What not to change (protected links, interfaces, configurations).

How to verify (compile, unit tests, automated tests, diff, performance metrics).

Failure handling (rollback plan, human gate, stop conditions).

Task execution records include modification details, reasons, and unchanged items, enabling traceability and replay.

7. Project Overview File and Automated Execution

When tasks proliferate, a project overview file serves as a navigation and status page, recording background, goals, constraints, milestone progress, and links to artifacts. Automation reads this overview to locate the next executable Task, runs it within the Harness + Loop, and writes back execution records and verification results.

8. Diff‑Driven Feedback Loop

After each deployment, shadow traffic runs the new and old flows side‑by‑side. Differences are collected, clustered, and analyzed to locate root causes. AI assists in generating repair suggestions, but human judgment decides whether to apply, gray‑release, or roll back the fix.

9. Results of the Refactor

The project achieved:

Delivery efficiency: total effort reduced from ~100 person‑days to ~30 person‑days (≈70% improvement).

Performance: main‑flow P50 latency dropped from 191 ms to 59 ms and P90 from 644 ms to 195 ms (≈70% reduction).

Resource savings: machine‑resource cost cut by about 45%.

Launch outcome: no QA involvement, smooth release without faults.

These gains were possible because the engineering constraints ensured behavior consistency, automated verification, and controlled release mechanisms.

Conclusion

AI coding’s value in large‑scale refactoring lies not in writing more code but in embedding engineering governance into a fine‑grained, continuously executable loop. Clear targets, bounded scopes, well‑defined Tasks, and automated verification let AI handle high‑frequency work while humans retain critical decision‑making, resulting in faster, safer, and more repeatable system evolution.