Zhihu's Database RAG Assistant: Cutting On-Call Load with Hybrid Search & MongoDB 8.3
Zhihu built a RAG-based database intelligent customer service using MongoDB 8.3's hybrid vector and full-text search, enabling developers to self-serve common DB issues while DBAs maintain a versioned, permission-controlled knowledge base with traceable answers and rollback capabilities.
Problem Background
Database teams frequently face repetitive queries from developers about connection timeouts, slow queries, Redis hotspots, TiDB DDL, and OceanBase resource configuration. Developers typically @DBA in chat groups, forcing DBAs to manually search across Wiki pages, PDF manuals, historical incident reports, and personal notes. The knowledge exists but is scattered across multiple sources with high search cost and no unified, traceable access path.
Project Goals
Serve developers: cover high-frequency, standardized DB issues to reduce repetitive consultations and ineffective on-call.
Serve DBAs: provide a single entry point to retrieve team knowledge without relying on memory.
Traceable answers: every response cites source documents, skill names, and sections for verification.
Controllable permissions: developers access only shared knowledge; DBAs access full knowledge including operational details and high-risk procedures.
Governable knowledge: document updates support preview, activation, and rollout to prevent immediate impact on production retrieval.
The first phase is a knowledge-based intelligent customer service that answers questions and provides guidance, not executing production changes directly.
Overall Architecture
The system comprises six core layers plus a continuous knowledge governance pipeline:
1. User & Interaction Layer
Frontend built with Streamlit provides a chat-like interface for developers and DBAs. Users can start new questions, continue conversations, and view accessible documents. Each request carries session context and user identity. In production, identity must come from a trusted login/JWT/gateway; the backend decides roles, never trusting a frontend-supplied role field.
2. API & Session Orchestration Layer
FastAPI exposes unified endpoints: /chat for Q&A, /documents for listing accessible docs, /conversations for history, and /health for status. This layer isolates the frontend from underlying models and retrieval logic, enabling future integration with Enterprise WeChat, DBA digital avatars, or other internal platforms.
3. RAG Retrieval & Answer Orchestration Layer
Core pipeline: user question → 1024-dim vector via embedding model → parallel vector search and full-text search in MongoDB → results fused via Reciprocal Rank Fusion (RRF). Three filters applied during retrieval: is_active (only activated versions), access_level (developer vs DBA), database_types (MySQL, TiDB, Redis, OceanBase, etc.). If a DBA Skill is hit, its global rules (safety, output, must-read) are automatically appended to avoid missing critical constraints.
4. Model Service Layer
Decoupled embedding and generation models. Embedding model maps documents and questions to the same 1024-dim semantic space; LLM reads retrieved context and generates structured Chinese answers. Both use OpenAI-compatible APIs, allowing switching between Aliyun Bailian, internal models, and internal Claude gateway, reducing vendor lock-in. If the knowledge base lacks coverage, the LLM falls back to general knowledge with an explicit disclaimer that the answer is not from internal sources (not real-time internet search).
5. MongoDB Data & Retrieval Layer
MongoDB handles three roles: knowledge_chunks collection stores chunks, metadata, and 1024-dim vectors; maintains full-text and vector indexes for semantic and keyword search; conversations collection stores history, answers, and citations. This eliminates the need for separate document DB, vector DB, and search engine, keeping original text, permissions, version status, and vectors in a single document.
6. Knowledge Production & Update Layer
Sources: PDF manuals, Markdown docs, DBA Skills, incident reports, personal notes. Ingestion pipeline: parse → chunk → tag with DB type and access level → embed → write new version to MongoDB (inactive) → human verification → activate. New versions default to is_active=false, only entering retrieval after preview and test-set validation. Rollback to previous version is supported if quality degrades, turning knowledge updates into a reviewable, recoverable release process.
End-to-End Question Answering Walkthrough
Example: Developer asks "TiDB DDL never finishes, what to check first?"
Receive question & identity: Streamlit sends question, session ID, DB type, user identity to FastAPI; backend loads conversation history for context.
Generate query vector: Same embedding model used at ingestion converts question to 1024-dim vector; same model and vector space ensure meaningful semantic distance.
Permission pre-filter: Retrieval applies access_level and is_active at search time, not post-filtering. Developer queries never see DBA-only chunks in candidate set.
Dual retrieval paths: Vector search ( $vectorSearch) finds semantically similar content (e.g., "online DDL stuck" matches "DDL never finishes"). Full-text search ( $search) finds exact terms like error codes (ERROR 9007), parameter names (tikv_gc_life_time), commands.
Fusion ranking via RRF: RRF score = Σ 1/(60 + rank) across both result lists. Chunks ranking high in both get boosted; single-path hits still have a chance.
Rule augmentation & context assembly: Top-K chunks plus any hit Skill's safety/global rules are assembled into model context with source, section, and content.
LLM answer generation: Prompt enforces: prefer internal sources, don't fabricate interfaces/parameters, distinguish KB conclusions from general knowledge, retain safety confirmations for high-risk ops.
Return answer, sources, save history: Response includes natural language answer plus source Skill, section, file. Q&A written to conversations for follow-up and audit.
Why MongoDB 8.3 as Retrieval Backbone?
Traditional RAG often requires separate relational DB, search engine, and vector DB, increasing sync, permission consistency, and ops overhead. MongoDB 8.3 consolidates documents, metadata, vectors, sessions, and search in one stack.
mongod vs mongot
mongodstores business documents: original text, embedding arrays, DB type, access level, version, activation status, conversation history. mongot is a separate Search/Vector Search process built on Apache Lucene, maintaining full-text and vector indexes. It syncs mongod changes via Change Streams and stores index segments in dedicated storage. Application connects only to mongod; when aggregation pipeline contains $search or $vectorSearch, mongod forwards to mongot. This separates transactional storage from search compute while keeping a unified data model and access entry.
Vector Search Mechanics
Each chunk gets a 1024-dim vector at ingestion, stored with text and metadata. Vector index uses cosine similarity; $vectorSearch performs approximate nearest neighbor (ANN) via HNSW (Hierarchical Navigable Small Worlds), avoiding full scan. numCandidates expands candidate pool before final Top-K, balancing latency and recall. Vector search excels at semantics but not exact matches (error codes, instance names, config params, SQL keywords where one character changes meaning), so it cannot stand alone for DB knowledge.
Full-Text Search Role
Full-text index covers chunk body, section titles, source names. $search performs lexical search with Lucene-based relevance scoring, ideal for error codes, command names, parameter names, DB object names, version numbers. Vector handles "semantic similarity"; full-text handles "literal precision"; they complement.
Hybrid Search Fit for DB Q&A
DB questions often mix natural language intent with precise technical entities (e.g., "TiDB scale-out caused Region imbalance, how to check schedule-limit?" needs both semantic understanding of "scale-out imbalance" and exact match of "schedule-limit"). Current implementation runs $vectorSearch and $search separately, fuses via RRF in Python — a verified path. MongoDB 8.3 also offers native $scoreFusion (GA in 8.3) for normalized, weighted fusion inside the aggregation pipeline, which can later replace application-side RRF and allow weight/rerank experimentation. Note: Search/Vector Search indexes update asynchronously via Change Streams (eventual consistency). A document write to mongod does not guarantee immediate searchability. Release flow must wait for index READY status and retrieval validation before activating a new version.
Permission Enforced at Retrieval Layer
DB knowledge may contain admin interfaces, incident handling steps, account policies, high-risk change procedures. Permissions must be applied during candidate generation, not just UI hiding. Two tiers: shared (dev + DBA) and dba_only (DBA only). Both vector and full-text searches apply permission filters at retrieval time. For external exposure, a trusted identity system maps real users to roles, and audit logs record querier, hit documents, answer sources, timestamps.
Continuous Knowledge Base Updates
Long-term RAG value depends on fresh, accurate, rollback-able knowledge. Versioning and activation state per document:
Updated doc creates new version, written inactive.
Real question set previews new version recall.
DBAs review answers, sources, safety prompts.
On pass, activate version switch.
If production quality drops, quick rollback to previous version.
Knowledge updates become like code releases: preview, acceptance, cutover, rollback — not "delete old, re-import".
Measured Benefits
Developers: Shift from "find people" to "self-serve first". Describe problem directly without knowing doc structure or which manual holds the answer. Common issues get sourced guidance instantly; complex cases escalate to DBAs.
DBAs: Move from repetitive answering to knowledge maintenance. Personal experience becomes reusable team assets. Time redirects to complex incidents, capacity governance, stability engineering.
Team: Unified, traceable knowledge outlet. Same question yields consistent answer regardless of who is asked. Source display, permission control, versioning, rollback clarify trustworthiness, currency, and usage boundaries.
Platform: Reusable DB intelligence capability. FastAPI encapsulates retrieval as a service; Streamlit is just the first entry. Future: Enterprise WeChat bot, DBA digital avatar, ticketing system, alert diagnosis workflows — all sharing the same knowledge foundation.
Key metrics to track: developer self-service rate; repeated on-calls deflected; mean time to first valid answer; answer source coverage & adoption rate; no-recall/low-quality-recall/manual-correction ratio; knowledge update-to-production cycle time.
Closing Thoughts
Database intelligent customer service is not just attaching an LLM to a doc catalog, nor replacing DBA judgment. It solves: how to organize scattered expertise, how to retrieve within correct permission boundaries, how to make answers verifiable, knowledge updatable, errors rollback-able. When developers self-serve common issues and DBAs instantly retrieve team-accumulated experience, on-call can return to tasks requiring true professional judgment. The shift from "hunting docs everywhere" to "asking directly" transforms DB knowledge from personal experience into team assets.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
