How Vector Databases Power Intelligent Medical Q&A Systems

Project Overview

The rapid development of artificial intelligence is reshaping healthcare, yet uneven distribution of quality medical resources and lack of preliminary health assessment tools cause wasteful consultations. The "Intelligent Diagnosis Q&A System" was created to let AI‑driven multi‑turn dialogues assist users in preliminary health evaluation, improving accessibility and diagnostic efficiency without replacing clinicians.

Two core challenges were identified: (1) semantic retrieval of massive medical knowledge (tens of thousands of entries) where traditional keyword search fails to capture relationships such as chest tightness ↔ angina; (2) maintaining contextual semantic matching across multiple dialogue rounds, requiring real‑time approximate nearest‑neighbor (ANN) search on high‑dimensional vectors.

Vector Database Fundamentals

A vector database stores high‑dimensional floating‑point vectors generated by embedding models (e.g., BERT, text-embedding-ada-002) and indexes them for similarity search using distance metrics such as cosine similarity or Euclidean distance. The workflow consists of three stages: (1) encode raw data into dense vectors; (2) build an index (common algorithms include HNSW, IVF, PQ) to balance precision and speed; (3) perform ANN search to retrieve the top‑K most similar vectors.

Advantages highlighted are strong semantic understanding, millisecond‑level retrieval on billion‑scale vectors, and natural fit for AI pipelines (RAG, semantic recommendation, image search). Disadvantages include weaker exact‑match capability, limited ACID guarantees, higher storage/computation cost for high‑dimensional data, and immature monitoring and backup tooling compared with relational databases like MySQL or PostgreSQL.

Application in the Intelligent Diagnosis System

After evaluating requirements, the team adopted a RAG+vector‑database architecture. Milvus 2.x was selected as the vector engine, while PostgreSQL handled structured business data, forming a hybrid vector‑relational storage.

Data flow : user symptom description → NLP service generates a query vector → Milvus performs ANN search to retrieve top‑K medical knowledge fragments → fragments are concatenated with the dialogue context to form a prompt → a large language model generates follow‑up questions or diagnostic suggestions → response returned to the user.

Knowledge base construction : 480,000 medical entries covering major specialties were cleaned and chunked into ~1.2 million fragments. The domain‑tuned embedding model bge-large-zh-v1.5 produced 1024‑dimensional vectors, achieving MRR@10 = 0.87 on a medical similarity benchmark, far surpassing a generic model (0.71).

Indexing strategy : For the 1.2 million vectors, an IVF_FLAT index was used for offline batch retrieval, while an HNSW index served online real‑time queries. Experiments showed HNSW delivered an average query latency of 18 ms for Top‑10 recall with 96.3% recall rate.

Collections design : Two Milvus collections were created— medical_knowledge (storing knowledge fragment vectors) and question_template (≈8,000 doctor‑reviewed follow‑up templates). The latter is partitioned by department and symptom category, enabling mixed vector‑scalar queries with retrieval latency under 12 ms.

System integration and performance tuning : A three‑node Milvus cluster with horizontal QueryNode scaling was deployed. Memory‑mapped files accelerated hotspot vector access. Redis caching of frequent query results achieved a 68% hit rate, further reducing Milvus query pressure. The overall system raised knowledge retrieval accuracy to 92% and improved user experience.

Summary, Reflections, and Future Work

The integration of Milvus and RAG resolved the core technical hurdles of semantic medical knowledge retrieval and multi‑turn dialogue grounding. The hybrid vector + relational architecture mitigated the precision loss of pure ANN search, while domain‑specific embedding fine‑tuning proved essential for high retrieval quality.

Future directions include exploring sparse‑dense hybrid retrieval (combining BM25 with semantic vectors) and further optimizing indexing strategies for complex medical terminology, aiming to push both recall and precision in increasingly demanding scenarios.