DeepSeek Open-Sources DeepInfer Inference Engine: 155T Tokens/Day, $38/GPU/Hour

Not Just a Model Release

On October 20, DeepSeek open-sourced DeepInfer , the inference scheduling layer it runs in production, along with the token codec component deepseek-recipe . The GitHub repository gained nearly 12,000 stars in 24 hours, topping the daily trending list, and now approaches 18,000 stars.

DeepInfer acts as an "operating system" for GPU/NPU clusters, providing optimized operators, parallel scheduling, and deterministic sampling for DeepSeek-series models. It supports both NVIDIA SM100 (Hopper) and Huawei Ascend 950 NPU hardware. The companion deepseek-recipe aligns token boundaries between training and inference; inconsistent tokenization would otherwise cause cache-hit-rate jitter and first-token latency spikes.

The V4.1-Flash-1017 model weights (484.62B parameters, multimodal) are released under the MIT license. DeepSeek's proprietary Search & Fetch API is priced at $0.75 per 1,000 calls. With the model, inference stack, and toolchain now all available, any cloud provider can follow this blueprint to drive per-unit costs down to near DeepSeek's level, pushing the price floor for inference even lower.

155 Trillion Tokens and $38 per GPU Hour

For the past two years the industry has competed on benchmark scores and leaderboard rankings. This release shifts the metric to "dollars earned per GPU per hour." The disclosed figures — 155 trillion tokens processed daily on a single cluster, translating to $38 of revenue per GPU per hour — answer two questions at once: the inference business model is viable and profitable, and that profitability stems from scheduling efficiency and throughput optimization.

The immediate implication for developers is straightforward: per-token pricing is likely to continue falling.