Comprehensive Cost Assessment of End-to-End RAG Systems

Overview

Targeted at technical and business decision‑makers building or operating Retrieval‑Augmented Generation (RAG) systems, the report quantifies the full‑stack operational cost, covering large‑model inference, vector databases, embedding model calls, public‑network bandwidth, and multimodal raw‑media storage.

Cost Structure

A production‑grade RAG system’s monthly cost can be split into five direct modules: LLM inference, vector database (compute + storage), embedding inference, network bandwidth, and infrastructure (monitoring, logging, load balancing, etc.).

Key Findings

LLM inference is usually the largest single expense, accounting for more than 40 % of total cost; model selection can change this by 5‑10×, making it the top optimisation priority.

Vector‑database cost is driven by compute resources (CU/AU/RU) rather than storage; quantisation (int8/ binary) typically saves over 50 % of compute and storage cost.

Embedding calls are a “small head”; even with tens of millions of daily queries, monthly cost stays in the low‑thousands of RMB.

In multimodal scenarios, raw‑media storage and outbound bandwidth can exceed the vector‑database cost and must be accounted for separately.

Cloud‑hosted vs self‑hosted trade‑off depends heavily on data scale and operational capability; small‑scale or unstable query patterns favour cloud free tiers, while >100 M vectors with an ops team often make self‑hosting cheaper.

Methodology

Data sources are divided into two credibility levels: unmarked entries are verified against official vendor pricing pages or cross‑checked by at least two independent third‑party sources; entries marked ※ come from third‑party price‑tracking sites and should be re‑validated before procurement. All prices reflect July 2026 market rates.

Cost formulas used:

Monthly cost = instance‑hour‑price × 730 + storage(GB) × storage‑price/GB/month

Embedding build cost = total‑documents × avg‑tokens/1 000 000 × model‑price

Query cost = daily‑queries × avg‑tokens/1 000 000 × model‑price × 30

Bandwidth cost = daily‑queries × response‑size(KB) ÷ 1 048 576 × 30 × unit‑price

Representative Scenarios

In a illustrative production workload (100 k daily requests, balanced‑size LLM deployment) LLM inference dominates the cost share (≈40 %). Vector database is the second‑largest component, while embedding and bandwidth together usually stay below 10 %.

Embedding pricing example: OpenAI text‑embedding‑3‑small costs $0.02 per M tokens (≈¥0.14). Indexing 100 M documents (500 tokens each) costs about ¥7,200 once, and 1 M daily queries (30 tokens each) cost roughly ¥130 per month.

LLM inference price snapshot (Claude series, July 2026): Haiku 4.5 – $1/​M input, $5/​M output; Sonnet 4.6 – $3/​M input, $15/​M output; Opus 4.8 – $5/​M input, $25/​M output. For 1 M daily requests (2 k input + 500 output tokens) monthly costs range from $525 (Haiku) to $2,625 (Opus).

Decision Matrix

Personal / prototype (<10 k vectors): use free tiers of any vector service.

Startup MVP (10 k–100 k vectors): Pinecone or Zilliz Serverless lowest tier.

Small production (1 M–10 M vectors): Serverless or lowest‑paid tier.

Mid‑scale (10 M–100 M vectors): Dedicated or self‑hosted after testing.

Large‑scale (>100 M vectors): self‑hosted preferred if ops team available; otherwise Dedicated.

Multimodal projects: explicitly budget for raw‑media storage and bandwidth, as they often become the dominant cost.

Optimization Recommendations

Model routing is the first priority: assign lightweight tasks to low‑cost models (e.g., Haiku) and reserve high‑capability models (Opus/Fable) for complex queries; combined with batch processing and prompt caching this can cut LLM inference cost by 5‑10×.

Quantisation before indexing (int8 or binary) should be the first step for vector‑database cost reduction, typically saving >50 % of compute and storage expenses.

For multimodal projects, calculate raw‑media storage and outbound bandwidth early, as they can surpass vector‑database costs.

Before final procurement, reproduce the report’s calculations with the vendor’s official pricing calculator to verify estimates.