How GPT‑5.6 Rewrote Its Own Kernel to Slash Service Costs by 20%

Self‑optimizing inference stack in GPT‑5.6 Sol

When GPT‑5.6 Sol was deployed, it autonomously rewrote and optimized the production GPU kernels that power inference. The model inspected real‑world traffic, identified load‑imbalance hotspots, and generated new routing strategies. It then produced rewritten kernels in Triton and Gluon that target operations suitable for pre‑computation, skipping, or parallel execution.

Two‑step methodology

Step 1: train a sufficiently strong model.

Step 2: use that model to improve everything that runs it—including the inference stack and underlying kernels.

Kernel rewrite and routing improvements

The model analyzed production traffic, uncovered previously ignored load‑imbalance, and tested alternative routing policies to keep GPU cards evenly utilized. It rewrote the low‑level kernels to eliminate unnecessary work and to expose parallelism.

Speculative decoding experiments

For speculative decoding, GPT‑5.6 Sol created a smaller draft model, designed and executed hundreds of architecture experiments on it, and monitored the training process for hardware failures or instability, intervening automatically when needed. This replaces the manual, months‑long effort of GPU engineers who would otherwise tune each configuration by hand.

Verification with FpSan

All generated kernels are passed through OpenAI’s open‑source floating‑point sanitizer (FpSan) to check structural and data‑flow correctness before deployment.

Quantitative outcomes

External service cost reduced by 20 %.

Token‑generation efficiency increased by more than 15 %.

Active researchers’ daily token output more than doubled compared with the GPT‑5.5 era.

Internal code‑inference compute share rose 100‑fold.

RSI (Recursive Self‑Improvement) Index scored 16.2 points higher than GPT‑5.5.

Closed‑loop workflow

The end‑to‑end loop now consists of traffic analysis → optimization proposal → code generation → experiment execution → fault handling → metric validation, with the model participating in each stage. Human engineers still define goals, select the production system, choose verification tools, and approve deployments.

Implications for competition

The 20 % cost saving allows OpenAI to handle roughly two‑thirds more requests on the same hardware, shifting the competitive axis from sheer GPU quantity to inference efficiency—how many tokens can be produced per GPU. GPT‑5.6 is offered in three tiers (Sol, Terra, Luna) that emphasize “same intelligence, fewer tokens,” positioning efficiency as a first‑class metric alongside model capability.

Code example

来源：新智元
本文
约2700字
，建议阅读
5
分钟
这回，AI开始为自己写代码了！