How Many GPUs Does Kimi K3 Need? Self‑Hosting vs API Cost Comparison

Model Size and Quantization

Kimi K3, released by Moonshot AI in July 2026, is a 2.8 trillion‑parameter mixture‑of‑experts (MoE) model that activates only about 104 billion parameters per inference, giving a low activation‑to‑total‑parameter ratio of 3.7 %.

Unlike most models trained at 16‑bit precision, Kimi K3 uses a native 4‑bit floating‑point format (MXFP4) and quantization‑aware training (QAT), so the model learns to operate effectively at 4‑bit precision without post‑training loss.

Memory needed for the full weight set can be estimated as: 2.8 trillion parameters × 4 bits ÷ 8 = 1.4 TB Running at 8‑bit or 16‑bit would double or quadruple this requirement.

Runtime Memory and GPU Requirements

For a single user request, KV cache (2–15 GB), activations (~30 GB), and runtime overhead (~30 GB) add to the 1.4 TB weight footprint, totaling roughly 1.5 TB of memory.

Consequently, a single H100 (80 GB) would need about 19 cards, while an H200 (141 GB) would need about 11 cards. More recent GPUs that support native MXFP4—Blackwell B200/B300 or AMD MI350X/MI400—are recommended.

The vLLM deployment guide suggests using 8 NVIDIA B300 or 8 AMD MI355X GPUs with tensor‑parallelism set to 8.

Cost Comparison: Self‑Hosting vs API

DigitalOcean’s serverless inference charges $3 USD per million input tokens and $15 USD per million output tokens, matching the official Kimi API pricing.

Running an 8‑GPU node (e.g., 8 MI350X) costs about $4.76 per GPU‑hour, or $38 USD per hour. A reserved GPU Droplet would cost roughly $27,800 USD per month regardless of usage.

To match this cost with the API, a user would need to generate about 1.8 billion output tokens per month (≈700 tokens / s continuously), which is far beyond typical single‑user workloads.

For heavy users (≈50 million tokens / month), API expenses cap at about $750 USD, while self‑hosting remains ~40 × more expensive for a single user.

When Self‑Hosting Becomes Viable

If a service serves 40 or more heavy users (each >50 million tokens per month) or runs dozens of high‑throughput agents, the GPU utilization rises enough for self‑hosting to become cost‑effective.

Additional considerations include data residency, latency consistency, control over the inference stack, and the custom Kimi K3 license, which permits commercial use but imposes conditions on large‑scale “model‑as‑a‑service” deployments.

Practical Deployment Guidance

Use tensor parallelism across 2, 4, or 8 GPUs; 8‑GPU configurations are the most realistic minimum.

GPU rentals are typically offered in 1‑ or 8‑GPU bundles, making a 6‑GPU setup impractical.

For workloads requiring <10 K token contexts, a single GPU can handle ~600 concurrent requests within a 600 GB memory pool.

DigitalOcean provides both serverless inference and dedicated GPU Droplets; the latter requires the user to manage the vLLM stack and supply Kimi K3 weights.