How Alibaba Cloud Optimizes Enterprise RAG: Key Techniques for AI Search

During the 2024 Alibaba Cloud Yúnxī Conference, senior AI Search researcher Xing Shaomin presented the enterprise‑grade Retrieval‑Augmented Generation (RAG) technology, explaining how it can improve decision support, content generation, and intelligent recommendation across core business scenarios.

Enterprise RAG Key Links

RAG (Retrieval‑Augmented Generation) combines search results with large language model (LLM) generation. The Alibaba Cloud AI Search Open Platform provides a RAG pipeline where red sections represent offline processes and blue sections represent online processes.

Enterprise RAG Effect Optimization

Document parsing : Supports various formats (PDF, Word, PPT, structured data) and extracts complex content such as tables and images.

Text slicing : Hierarchical and multi‑granularity slicing (paragraph‑level, sentence‑level) and semantic slicing using LLMs before vectorization.

Hybrid retrieval : A unified vector model produces dense and sparse vectors for mixed retrieval and re‑ranking.

NL2SQL : Converts natural‑language queries into SQL for precise database lookups; also supports graph queries.

LLM rerank : Initially used Bge‑reranker; later replaced by a Qwen‑based rerank model, improving effectiveness by ~30%.

Enterprise RAG Performance Optimization

VectorStore CPU graph algorithm : Optimized HNSW construction and search, achieving roughly 2× performance over peers.

VectorStore GPU graph algorithm : Nvidia‑based implementation delivering 3‑6× speedup on T4 GPUs and up to 60× on A100/H100 for high‑QPS workloads.

Large‑model inference acceleration : Caching, quantization, and multi‑GPU parallelism enable 14B Qwen to generate 200‑token answers in 1‑3 seconds and 72B Qwen in ~4 seconds.

Enterprise RAG Cost Optimization

To reduce fine‑tuning costs, Alibaba Cloud adopted LoRA‑based methods, allowing dozens of customer models to share a single GPU card, cutting monthly expenses from ~4000 CNY to ~100 CNY (a 40‑fold reduction). Both single‑card and multi‑card LoRA strategies are supported.

Enterprise RAG Application Practice

The AI Search Open Platform exposes the discussed capabilities as micro‑services (document parsing, vectorization, NL2SQL, LLM Agent, evaluation, training, inference, etc.) accessible via APIs or SDKs (Alibaba Cloud, OpenAI, LangChain, Llamalndex). It supports both Havenask and Elasticsearch engines, with upcoming support for GraphCompute and Milvus.

A unified multimodal data management layer handles unstructured documents (PDF, Word, PPT) and connects to data lakes, databases, OSS, HDFS, reducing data migration costs.

Multi‑modal search and multi‑modal RAG scenarios are built by chaining services such as vectorization and image understanding. The offline flow processes PDFs into Markdown, slices them, and vectorizes; the online flow performs initial retrieval, summarization, LLM reasoning, secondary retrieval, and final synthesis using agents.

OpenSearch LLM edition enables a three‑minute RAG service deployment, supporting NL2SQL‑based table Q&A with structured outputs and recommendations.