How to Build a Financial RAG Solution with Alibaba PAI: Step-by-Step Guide

Overview

Retrieval‑Augmented Generation (RAG) combines information retrieval with generative AI to provide more accurate and relevant answers, which is especially valuable in domains such as finance where precise information is required. This guide shows how to build a financial‑focused RAG solution on Alibaba’s PAI platform.

Prerequisites

Activate PAI pay‑as‑you‑go and create a default workspace.

Create an OSS bucket to store training data.

Provision a Milvus vector database instance.

1. Prepare the Dataset

Collect domain‑specific documents (e.g., financial news PDFs), ensure they follow PAI‑Designer’s format requirements, and upload them to the OSS bucket.

wget https://atp-modelzoo-sh.oss-cn-shanghai.aliyuncs.com/release/solutions/rag/data/%E9%87%91%E8%9E%8D%E6%96%B0%E9%97%BBpdf.zip

2. Deploy LLM and Embedding Models

In the Model Gallery, select a large language model that supports instruction tuning (e.g., 通义千问2.5‑7B‑Instruct ) and a Chinese embedding model (e.g., bge‑large‑zh‑v1.5 ). Deploy both and record their VPC URLs and tokens.

3. Create Service Connections in LangStudio

In LangStudio, create connections for the LLM, the embedding model, and the Milvus vector store using the URLs and tokens obtained in the previous step.

4. Build the Knowledge Base with PAI‑Designer

Read data from the OSS bucket.

Parse and split the text into chunks.

Generate embeddings for each chunk.

Store the embeddings in Milvus.

The workflow can be visualised as:

5. Construct the RAG Application in LangStudio

Create a new application flow from the RAG template, then configure the following nodes:

rewrite_question : rewrites user queries; link to the LLM connection created earlier.

retrieve : retrieves relevant documents from Milvus; specify the vector store, index name, and embedding model connection.

threshold_filter : filters retrieved documents based on similarity score.

generate_answer : generates the final answer using the LLM connection.

After configuring the flow, start the runtime, choose a machine type, and deploy the RAG service.

Case Comparison (Financial vs. Medical)

Two example tasks demonstrate the benefit of RAG. In the financial risk‑analysis task, the plain LLM answer is verbose and uncertain, while the RAG‑enhanced answer provides a concise, evidence‑based response. Similar improvements are shown for industry‑trend analysis and trade‑statistics queries.