Deploy FastGPT Locally: Step‑by‑Step Docker & Source Code Guide for RAG AI

Background

Artificial intelligence (AI) is extremely popular, and many internal systems want to use GPT for basic Q&A and analysis. Directly asking GPT without providing domain knowledge fails, and token limits make maintenance difficult.

Retrieval‑Augmented Generation (RAG) stores knowledge in a vector database and retrieves the most similar answers based on query similarity, which is a popular solution.

RAG Workflow

Retrieval stage : Given a query, the system first retrieves relevant documents or fragments from a large corpus using information‑retrieval techniques such as vector search.

Generation stage : After retrieval, the generation model uses the retrieved texts to produce the final answer, combining query and context for more accurate output.

RAG provides richer background knowledge, especially for complex or domain‑specific tasks, and is widely used in QA, dialogue generation, and text summarization.

Platform Introduction

FastGPT is an open‑source knowledge‑base Q&A system based on large language models. It can set opening statements, annotate expected answers, and adjust many response parameters, offering more comprehensive functionality and better retrieval performance.

Import documents or existing Q&A pairs for training.

Automatic data preprocessing with multiple import methods (PDF, Word, Markdown, CSV, etc.).

FastGPT automatically preprocesses, vectorizes, and segments text, saving manual training time.

OpenAPI integration for external platform calls.

Source code: https://github.com/labring/FastGPT

Deployment Options

Two main approaches: deployment based on source code (customizable) and deployment based on Docker image (simple).

Docker Image Deployment

Docker‑compose is used to start FastGPT and its dependent services (PostgreSQL, MongoDB). The following steps install Docker and docker‑compose:

# Install Docker
curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun
systemctl enable --now docker

# Install docker‑compose
curl -L https://github.com/docker/compose/releases/download/v2.20.3/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

docker -v
docker-compose -v

Create a directory, download configuration files, and start the services:

mkdir fastgpt
cd fastgpt
curl -O https://raw.githubusercontent.com/labring/FastGPT/main/projects/app/data/config.json
curl -o docker-compose.yml https://raw.githubusercontent.com/labring/FastGPT/main/files/docker/docker-compose-pgvector.yml

docker-compose up -d

FastGPT deploys multiple services, including PostgreSQL for vector storage and MongoDB for business data.

Vectors represent text as numeric lists; similarity is measured by distance, similar to finding related books in a library.

Adjust OPENAI_BASE_URL and CHAT_API_KEY in the docker-compose.yml file (e.g., OPENAI_BASE_URL=https://oneapi.example.com/v1).

Source Code Deployment

Install pnpm, install dependencies, and start the application:

npm install -g pnpm
pnpm i
cd projects/app
pnpm install
pnpm start

Copy .env.template to .env.local and modify the following variables: MONGODBURI, PGURL, OPENAI_BASE_URL, CHAT_API_KEY.

If you prefer source deployment but lack existing MongoDB or PostgreSQL instances, first deploy them via Docker (as described above) and then reference their connection URLs in .env.local.

Our internal deployment platform (Moon) builds a Docker image from a custom buildDockerfile and runs it as a Kubernetes pod.

FROM node:18.15-alpine AS builder
COPY . /usr/src/app
WORKDIR /usr/src/app
RUN npm install -g pnpm
RUN pnpm i
ENTRYPOINT ["sh","-x","/usr/src/app/_appconfig/start.sh"]

The start script ( start.sh) launches the FastGPT service:

#!/bin/bash
cd /usr/src/app/projects/app; pnpm install; pnpm start

OpenAPI External Usage

After logging in, apply for a token and include it in the request header.

POST http://10.1.9.40:3000/api/v1/chat/completions
{
    "chatId": "abcd",
    "stream": false,
    "detail": false,
    "messages": [
        {
            "content": "澳门面积多大",
            "role": "user"
        }
    ]
}

Parameters: chatId maintains context when non‑null, stream controls token‑by‑token streaming, and detail toggles detailed output.

Summary

FastGPT can be set up easily with default configurations; the main maintenance tasks involve MongoDB (business data) and PostgreSQL (knowledge vectors). Using custom databases or changing ports adds complexity and may cause issues. The platform already stores over 50 k knowledge entries, so backup strategies are essential. Upgrading the Docker image is straightforward, while source‑code upgrades require more careful handling.