How to Build a High‑Performance RAG System with Milvus on Alibaba Cloud PAI

Background

With the rapid development of AI, large language models (LLMs) excel in text and image generation but suffer from domain knowledge gaps, outdated information, and hallucinations. Retrieval‑Augmented Generation (RAG) mitigates these issues by incorporating external knowledge bases, improving accuracy and personalization.

RAG Architecture

The core of RAG consists of retrieval and generation. Retrieval relies on efficient vector search engines such as Faiss, Annoy, HNSW, and the open‑source Milvus system, enabling fast and precise similarity search over large datasets.

Prerequisites

A Milvus instance with public network access (see Milvus quick‑start guide).

An Alibaba Cloud PAI (EAS) workspace (see PAI activation guide).

Usage Limits

Milvus instances and PAI (EAS) must reside in the same region.

Operation Flow

Step 1: Deploy a RAG System via PAI

Log in to the PAI console and select the target region.

Navigate to Model Deployment > Model Online Service (EAS) and enter the workspace.

Click Deploy Service and choose Large Model RAG Dialogue System .

Configure key parameters (others can use defaults):

Service Name : custom name.

Model Source : default open‑source public model (e.g., Qwen1.5‑7B).

Model Category : select appropriate model.

GPU Resource : choose suitable GPU configuration.

Vector Store : select Milvus, set collection name, internal address, proxy port, root account and password.

Collection Deletion : choose True to replace existing collection or False to append.

VPC, Switch, Security Group : use the same network settings as the Milvus instance.

After deployment, the service status changes to Running , indicating success.

In the service page, click View Web Application to open the Web UI.

Step 2: Use Milvus Vector Retrieval in the Web UI

Test connectivity: in the Settings tab, click Connect Milvus . A successful connection shows “Connect Milvus success”.

Upload data: in the Upload tab, upload TXT or HTML knowledge base files. Example message after upload:

Upload 1 files [ PAI.txt, ] Success!

Perform vector search: in the Chat tab, select RAG (Retrieval + LLM) and run queries.

For further assistance, join the Milvus‑Version user DingTalk group (ID: 59530004993).