Hy4 Preview: Tencent's 770B Open-Source LLM Boosts Throughput 31.8% and Reaches Top‑5 Arena Rank

Tencent announced the Hy4 preview, a next‑generation large language model with a total of 770 billion parameters, 49 billion activation parameters per inference, and support for up to 1 million tokens of context, targeting code, office, gaming, and scientific research scenarios.

The model uses a mixture‑of‑experts (MoE) backbone of 78 layers; the first layer is a dense feed‑forward network, while the remaining 77 layers each contain 256 routing experts and one shared expert, with each token activating eight routing experts plus the shared one. It incorporates a native MTP layer for decoding, Gated DSA attention, IndexCache for cross‑layer sparse index reuse, and iHC residual paths to enhance long‑context computation, decoding efficiency, and deep information flow.

Through a round of inference‑infrastructure optimization—identifying bottlenecks, applying operator fusion and communication optimizations—the end‑to‑end throughput increased by 31.8% compared with the baseline.

Compared with the previous Hy3 preview (295 B total parameters, 21 B activation, 256 K context), Hy4 expands both model capacity and context window dramatically.

Model weights and an FP8 version have been open‑sourced under the Apache 2.0 license. Relevant resources include the official blog, Hugging Face repository, and GitHub code. API pricing is ¥6 per million input tokens, ¥18 per million output tokens, with cache‑hit input at ¥0.3 per million tokens, and a two‑week free trial for WorkBuddy and CodeBuddy.

Benchmark results show substantial gains: Terminal‑Bench 2.1 rose from 70.8 to 85.4, DeepSWE from 28.0 to 64.3, Toolathlon‑Verified from 56.2 to 74.1, BioMysteryBench from 54.9 to 71.3, and GPQA Diamond from 90.9 to 92.3. On the Arena.ai Code Arena leaderboard, Hy4 scored 1633 points (AutoEval), placing roughly 5th overall and jumping 115 points from Hy3’s 31st position, now ranking about 3rd among open‑source models.

In an internal blind test, 163 Tencent experts evaluated 203 engineering tasks, giving Hy4 an average score of 2.99 out of 4, slightly ahead of GLM 5.3 (2.92) and Kimi K3 (2.94).

The model’s capabilities span software engineering (understanding requirements, planning changes, debugging, and improving UI/UX), complex office tasks (cross‑file information extraction, data and financial analysis, document generation), game development (generating playable prototypes in Unity or Unreal with iterative refinement), and scientific research (AI R&D, molecular dynamics, condensed‑matter physics, and foundational mathematics).