How Our In-House AI Agent Scaled to Handle 70% of Tech Support: A Six-Month Review

In May 2025 the infrastructure team delivered the first version of an AI platform, creating an incubator for AI digital employees. The team needed a solution for the daily flood of repetitive, low‑efficiency queries from various departments, but faced a lack of business‑value consensus for early‑stage AI products and strict resource‑allocation rules.

Incremental Intelligence Path

The design premise was that the AI’s business boundary must be rooted in existing system data, while delivery capability depends on the combination of data and large‑model inference. The AI is not meant to solve generic Q&A but to accurately understand system usage, business processes, and policy rules, and to provide traceable execution. Consequently, the agent serves as an entry and interaction layer built on three foundations:

System data: real documents, process states, form fields, permissions, and organizational structure to ensure answers are "grounded".

Domain knowledge base: operation guides, common cases, policy documents to ensure answers are "consistent".

Process and permission system: codified "what can be done, who can do it" rules to guarantee controllability and auditability.

The ultimate goal is not a "universal Q&A" bot but an intelligent gateway that conveys accurate information, automates standard workflows, and escalates non‑standard issues in a closed loop.

Pragmatic Implementation Strategy

Following a pragmatic principle of tackling the highest ROI tasks first, the team leveraged historical Feishu multi‑dimensional tables and bot‑derived problem‑solution data. Requirements were ranked by "frequency × time cost × risk", leading to two priority categories:

High‑frequency consulting questions that are repetitive, have volatile answers, and affect many collaborators.

Time‑consuming operational questions that involve many steps, are error‑prone, and incur high rework costs.

Rapid progress was possible thanks to a long‑standing "question closure" mechanism and years of review experience that allowed the team to extract common patterns, create reusable standard answers and operation guides, and clearly separate standard from non‑standard processes, defining the automation boundary.

Tiered Support System

Level 1 – General capabilities (open to all): standard answers, basic operation guides, troubleshooting FAQs, and navigation to process entry points. Characteristics: low risk, reusable, broad coverage.

Level 2 – Controlled capabilities (role‑specific): change operations, sensitive information handling, and permission‑restricted actions. Implemented via capability encapsulation, permission checks, operation logging, and audit trails.

This structure enables quick self‑service for common issues while ensuring that privileged actions remain governed and traceable.

In July 2025 the AI product was rolled out to business teams as a Feishu bot.

Outcomes and Reflections

After six months of continuous refinement, the AI evolved into a massive agent comprising more than 120 AI tools, 12 MCP groups, and a large knowledge base. It now independently handles over 70% of technical‑support work, aligning well with the original expectations.

However, technical debt accumulated in parallel. As tools and knowledge layers stacked, the agent’s workflow orchestration fell into a "high‑density mesh" with non‑linear complexity growth. Generalized support intents caused an explosion of logical branches, making each new feature a potential source of unpredictable chain reactions and raising maintenance and governance costs.

From an engineering standpoint, the implementation resembles "code sprawl": numerous prompts and versions coexist, with 20‑30% overlap or conflict among developers’ prompts, leading to "personality splits" when the model encounters edge cases. Minor prompt tweaks can cause dramatic output swings, highlighting a serious maintainability issue.

The six‑month exploration proves AI’s huge potential in business domains but also sounds a warning: standardizing AI engineering and ensuring maintainability will be the core challenges for the next phase.