Build a Custom LLM Chatbot with Hologres Vector Store and LangChain

Background

Large language models (LLMs) can be applied across industries. Prompt‑tuning can customize a chatbot for a specific vertical by providing domain‑specific corpora, leveraging the model’s reading comprehension and information integration abilities.

Hologres is a one‑stop real‑time data warehouse engine that integrates with Alibaba’s high‑performance vector computation library Proxima, offering efficient vector operations.

By using Hologres as a real‑time vector store, custom corpora can be stored as vectors and fed to an LLM, enabling rapid construction of a domain‑specific chatbot.

Prerequisites

Any LLM that can be accessed via API.

A Hologres instance (free trial available).

Setup

Environment preparation

conda create --name chatbot python=3.8
conda activate chatbot
pip install langchain psycopg2-binary torch transformers modelscope bottle
pip install <LLM‑related‑dependencies>

Obtain an API key for the chosen LLM.

Clone the code and sample data:

git clone https://github.com/aliyun/alibabacloud-hologres-connectors.git

Running the chatbot

Configure the model in holo-chatbot/chatbot.py (line 107) by setting chat_model_name to the desired model.

# Choose LLM
from langchain.chat_models import <chat_model_name>
chat_model = <chat_model_name>(api_key='xxx', api_base='xxx', temperature=0.1)
bot = Chatbot(chat_model, args.clear, args.no_vector_store)

Start the configuration wizard and launch the chatbot:

cd alibabacloud-hologres-connectors/holo-chatbot
python generate_config.py
python chatbot.py -n

Example interactions

Human: What is Hologres?
Chatbot: Hologres is a cloud‑native analytical data warehouse by Alibaba Cloud, supporting massive real‑time ingestion, updates, and analytics with high‑performance SQL and PostgreSQL compatibility.

Human: How does Hologres accelerate jsonb queries?
Chatbot: Hologres provides a Gin Index for jsonb columns, allowing fast location of rows containing specific key‑value pairs. You can create it with:
CREATE INDEX idx_gin ON my_table USING gin(my_jsonb_column);
The index speeds up jsonb operators such as ->, ->>, #>, #>>.

Human: What is column‑store jsonb in Hologres?
Chatbot: Column‑store jsonb stores each key‑value pair of a jsonb document in separate columns, enabling faster queries and better compression because only the needed columns are scanned.

Vector data preparation

Create a Hologres database and configure connection parameters (HOLO_ENDPOINT, HOLO_PORT, HOLO_USER, HOLO_PASSWORD, HOLO_DATABASE) in config/config.yaml.

Prepare a CSV corpus with title and content columns, then convert it to 768‑dimensional embeddings using the CoRom Chinese embedding model and store them in Hologres: python chatbot.py -l Verify the vectors are stored by querying the langchain_pg_embedding table.

Verification

Run the chatbot without vector store ( -n) and with vector store to compare answers; the domain‑tuned model provides richer, more accurate responses about Hologres features.

Optional DingTalk integration

Deploy the DingTalk server script dingding_server.py and configure tokens in config/config.yaml to enable the chatbot in a DingTalk group.

Start the DingTalk service:

# This script runs the DingTalk bot in the background
python dingding_server.py > log.txt 2>&1 &

After deployment, mention the bot in a DingTalk group to start chatting.