Ctrip’s New ChatBI Paradigm Using Harness and Skill

As large‑language‑model (LLM) capabilities improve, the limitations of conventional data products become more apparent: inconsistent metric definitions, difficulty extracting data, and reliance on manual root‑cause analysis. The author notes that these issues persisted because AI’s semantic understanding and reasoning were not yet mature, and only with the recent maturity of LLMs does a practical solution become feasible.

Based on this insight, Ctrip built a production‑ready ChatBI solution. The architecture features a Multi‑Agent collaboration framework where each sub‑agent has a dedicated role, a memory‑management layer that separates short‑term context from long‑term preferences, and a Harness‑based tool orchestration layer that, together with a Skill management component, standardizes data retrieval and attribution. For the underlying LLM, the team compared Claude SDK with Ali Agent Scope. At the infrastructure level, they constructed a metric engine and a topic‑governance system, organized knowledge documents, and implemented concrete methods for metric identification and dimension extraction. Quality is monitored through accuracy tests, automated testing, an active clarification mechanism, and a funnel‑style link diagram, with results graded by confidence levels to ensure reliable output.

The solution will be presented at an AI meetup on July 10, 14:00‑17:30 in Shanghai’s Ant S‑Space, co‑hosted by OceanBase and DataFun, where Ctrip experts will share detailed insights. Interested participants can register for free by scanning the QR code.