Deploy Qwen3-4B on FlagOS with OpenClaw: A Complete Step‑by‑Step Guide

Background

Traditional AI deployments rely on public‑cloud services, which raise data‑privacy concerns and incur high API costs. To enable 24/7 local AI agents, enterprises are moving toward private, on‑premise model serving. FlagOS, an open‑source AI system stack supporting heterogeneous chips, together with Tencent Cloud HAI, provides a practical path for this transition.

1.1 Install Qwen3-4B‑hygon‑flagos

Locate the Qwen3-4B‑hygon‑FlagOS image on the HAI community platform and pull it using Docker. First, install the ModelScope client to download the model weights.

pip install modelscope</code><code>modelscope download --model Qwen/Qwen3-4B --local_dir /share/Qwen3-4B

Pull the Docker image:

docker pull haihub.cn/baai/flagrelease_hygon_qwen3:v1.0.0

Start the container (modify the container name if needed):

# Container Startup
docker run -it \
 --name=flagos \
 --network=host \
 --privileged \
 --ipc=host \
 --shm-size=16G \
 --memory="512g" \
 --ulimit stack=-1:-1 \
 --ulimit memlock=-1:-1 \
 --cap-add=SYS_PTRACE \
 --security-opt seccomp=unconfined \
 --device=/dev/kfd \
 --device=/dev/dri \
 --group-add video \
 -u root \
 -v /opt/hyhal:/opt/hyhal \
 -v /share:/share \
 haihub.cn/baai/flagrelease_hygon_qwen3:v1.0.0 \
 /bin/bash

Enter the container and launch the service:

docker exec -it flagos bash</code><code>flagscale serve qwen3

1.2 Install and Configure OpenClaw

Clone and build OpenClaw from source:

git clone https://github.com/openclaw/openclaw.git
cd openclaw
pnpm install
pnpm ui:build # auto‑installs UI deps on first run
pnpm build
pnpm openclaw onboard --install-daemon
pnpm gateway:watch

Set the model provider to use the local vLLM inference service:

pnpm openclaw config set 'models.providers.vllm_local' --json '{
  "baseUrl": "http://1.15.51.106:9033/v1",
  "apiKey": "anykey",
  "api": "openai-completions",
  "models": [
    {"id": "Qwen3-4B-hygon-flagos", "name": "Remote Model"}
  ]
}'

Merge and activate the configuration:

pnpm openclaw config set models.mode merge
pnpm openclaw models set vllm_local/Qwen3-4B-hygon-flagos

Verify the model switch:

pnpm openclaw configure

1.3 Configure QQ Channel

Follow the documentation at https://cloud.tencent.com/developer/article/2626045 to replace the placeholder ID and secret with your own credentials. Then start the OpenClaw gateway: pnpm openclaw gateway After the gateway is running, you can chat with the QQ bot either in a private conversation or a group. Successful AI‑driven replies confirm that the OpenClaw‑QQ integration works.

Trend Outlook

Testing Qwen3-4B‑hygon‑flagos in the OpenClaw‑QQ scenario shows that small models (4B parameters) can now serve as reliable agents, handling instruction understanding, tool invocation, file operations, and workflow control. The real bottleneck shifts from model size to system integration, permissions, and API design, making 4B‑level models a practical default for on‑premise agents.

Less is more—FlagOS is the key to unlocking efficient, private AI agents.