Why a General ChatBot Fails in Vertical Apps and How a Sub‑Assistant Matrix Works

Three Typical Pitfalls of Copying ChatGPT in Vertical Apps

Most AI‑assistant projects I have seen shrink ChatGPT into a tiny widget inside an app, presenting a generic greeting and three suggested questions. Users close it because they did not open the app for general Q&A.

In the Chao News app we tried the opposite: a single general assistant plus five vertical sub‑assistants. After a trial period we reached a counter‑intuitive conclusion – a generic ChatBot is essentially a false demand in vertical apps; users only adopt the specialized assistants.

Pitfall 1 – Shrinking ChatGPT into the app. The logic sounds reasonable – “what ChatGPT can do, we let users do inside the app” – but users of a news or civic‑service app are not there to ask generic questions. If they need to know “what is photosynthesis”, they will open a dedicated large‑model app instead of navigating a news app.

Pitfall 2 – Believing a unified entry is better. A single dialog box cannot cover fragmented daily tasks such as checking a government document in the morning, booking a medical appointment at noon, captioning a photo in the afternoon, and reporting a local incident at night. Users do not know how to start the conversation, so they abandon it.

Pitfall 3 – “Start with a general bot, then add verticals”. In practice users form a negative impression after the first generic interaction – answers are vague, inaccurate, and do not satisfy their concrete needs. That impression blocks later use of any specialized assistant.

Our data shows that conversation rounds are high in dedicated large‑model apps (e.g., Doubao, Kimi) but drop sharply to one or two rounds when the same entry is placed inside a vertical app. The gap stems from differing user expectations, not from product quality.

Ask Yourself Three Questions Before Building a Matrix

1. Is the app single‑scenario or multi‑scenario? If you can list more than five distinct user goals within a minute, the app is multi‑scenario and may benefit from a matrix. Pure tools or e‑commerce apps are usually single‑scenario and need only one assistant.

2. Do users come with concrete tasks or just browsing? Task‑driven users map directly to a sub‑assistant; casual browsers only need a generic summarizer and do not require a full matrix.

3. Can your AI capability create professional differentiation? If a sub‑assistant would answer no better than ChatGPT, abandon it. Differentiation can come from exclusive data (e.g., local government files), task‑oriented flows (e.g., incident reporting), or strict compliance constraints (e.g., medical advice).

Chao News Matrix Architecture and Pitfalls

The final architecture consists of one general assistant for intent routing and five vertical assistants:

Document Search – serves civil servants needing official documents; requires closed data sources and precise retrieval by agency, year, and document number.

AI Reporting – guides users through multi‑step incident reporting, prompting for time, location, participants, media, and generating a structured draft for editors.

AI Photo Companion – generates news‑style captions by recognizing image elements and combining them with a local news corpus; the local corpus provides a style that generic models lack.

AI Diagnosis – enforces medical constraints (no diagnoses, no drug dosage, mandatory 120‑call for emergencies) via hard‑coded prompts and post‑processing filters.

Policy Q&A – answers citizen questions about local policies; maintains region‑specific knowledge bases with audit, source attribution, and freshness.

The general assistant does only fallback and routing; it never claims to answer everything.

Pitfall 1 – Cognitive split. Six AI entry points confused users. Solutions: let the general assistant route intents, add concise one‑sentence labels to sub‑assistant icons, and trigger context‑aware prompts (e.g., “Want to report this incident?” on a news detail page).

Pitfall 2 – Capability overlap. Overlapping answers (e.g., “Hangzhou settlement policy”) caused inconsistency. We defined a boundary protocol: each sub‑assistant has a primary responder, a fallback, and a rejection rule; the general assistant arbitrates conflicts. Regular evaluation sets detect inconsistencies for prompt or routing adjustments.

Pitfall 3 – Scaling cost. Adding a new vertical required new prompts, RAG pipelines, and evaluation sets, leading to exponential effort. We introduced a standardized onboarding template requiring role definition, data source, constraints, output format, a baseline Q&A set (≥50 pairs), and an operations SOP. Core capabilities (model calls, RAG, sensitive‑word filtering, logging) are shared across assistants, keeping marginal cost low.

Team requirements: at least two product managers – one overseeing the general routing and overall experience, another focusing on depth of each vertical assistant. Each sub‑assistant needs its own evaluation set and a dedicated data‑maintenance owner; otherwise the knowledge base becomes stale.

Final takeaway: building AI assistants is a product‑architecture challenge, not merely an API‑integration task. Clarify what AI should accomplish inside your app, then cut or keep components accordingly.