How Much Does Deploying GLM‑5.2 Locally Cost? A Detailed Cost Breakdown

Model characteristics

GLM‑5.2 is a Mixture‑of‑Experts (MoE) model with a total of ~743 billion parameters. For each generated token only ~39 billion parameters are activated, and the model natively supports a 1,048,576‑token (1 M) context window.

Two precision variants are released:

FP8 weight file size: 756 GB

BF16 weight file size: 1.51 TB

All expert weights must reside in GPU memory; they cannot be streamed from disk during inference.

GPU memory requirements (vLLM recommendations)

FP8: minimum 893 GB GPU memory

BF16: minimum 1,786 GB GPU memory

An NVIDIA DGX H200 server with 8 × H200 SXM GPUs provides 1,128 GB HBM3e memory, satisfying the FP8 requirement but not the BF16 requirement.

Hardware specification of a single 8‑GPU H200 node

8 × NVIDIA H200 SXM GPUs (141 GB HBM3e each, total 1,128 GB)

GPU memory bandwidth: 4.8 TB/s

NVSwitch inter‑GPU bandwidth: up to 900 GB/s

Dual‑socket Intel Xeon 8480C CPUs

2 TB system RAM

2 × 1.92 TB system SSDs

8 × 3.84 TB NVMe data drives

8 × ConnectX‑7 400 Gbps network adapters

Maximum power draw: ~10.2 kW

Cost analysis

Based on recent market pricing (early 2024), a complete 8‑GPU H200 server costs roughly 3.5 million CNY. Price increases for memory, storage, and networking push the cost to about 3.5 million CNY for a single node.

FP8 deployment:

Single 8‑GPU H200 node meets the 893 GB memory requirement, allowing the model to be loaded.

Practical context length starts at ~128 K tokens; larger contexts increase KV‑cache usage and reduce concurrent request capacity.

FP8 with full 1 M‑token context:

Single node can load the model but cannot comfortably serve high‑concurrency 1 M‑token requests.

Dual‑node design (two 8‑GPU H200 servers) with a 200 GB‑class RAM‑switch, optical modules, and cabling provides sufficient memory bandwidth and inter‑node connectivity for stable 1 M‑token service.

Cost breakdown for the dual‑node 1 M‑context FP8 deployment:

Two 8‑GPU H200 servers: ~7 million CNY

High‑speed networking (RAM‑switch, optics, cables): ~0.3 – 1 million CNY

Total: ~7.3 – 8 million CNY

BF16 deployment:

Requires at least two 8‑GPU H200 servers to meet the 1,786 GB memory requirement.

All other costs (hardware, networking) are roughly double the FP8 figures, leading to a total budget of ~7.3 – 8 million CNY for basic dual‑node deployment and >10 million CNY when adding concurrency or high‑availability requirements.

Power consumption for a dual‑node configuration exceeds 10 kW, necessitating high‑density racks and appropriate power distribution units.