Post‑Mortem of an AI‑Generated Flash‑Sale System Failure at Ant Internal Network

A few days ago Ant's internal network experienced a crash during a small spring‑tea flash‑sale activity, initially blamed on the AI‑generated low‑code system that built the service.

Investigation revealed that while the AI‑generated business logic was correct, the overall development lifecycle—including testing, building, deployment, and especially operational reliability—still required human expert intervention.

Ant reports that over 60% of its engineers have used the internal code‑large model, with about 10% of submitted code generated by AI; the acceptance rate of AI‑generated code is 30% overall and reaches 60% for automatically created unit tests.

The root cause of the outage was a classic avalanche effect: the database was provisioned for a maximum of 1,000 QPS but the flash‑sale peaked at 2,000 QPS, causing latency to increase hundred‑fold, the application to hang, and users to see white screens. Lack of capacity planning, rate‑limiting, and proper scaling led to the failure.

The incident demonstrates that AI can automate business‑logic coding but cannot replace programmers for deployment, performance tuning, and operations; human involvement remains essential for system reliability.

Future directions include automating capacity planning and best‑practice insertion (e.g., replacing row‑locks with Redis atomic updates), adopting cloud‑native auto‑scaling, integrating middleware to encapsulate optimal patterns, employing DBA‑style automated SQL optimization, and even training an AI agent to handle pre‑deployment checks, budgeting, and scaling actions.

A Q&A section answers three key questions: why the flash‑sale failed despite low internal traffic, whether AI‑written code caused bugs, and why resources were spent on a seemingly trivial internal system.

In conclusion, the team acknowledges that focusing solely on AI‑assisted development without robust operational automation caused the outage, and commits to improving reliability through better ops automation and balanced AI‑human collaboration.