Chinese AI Beats OpenAI and Anthropic with 86.3% Success on CyberGym

AI‑generated code is no longer novel; the real challenge is embedding AI into large, unfamiliar codebases so it can act as a continuous security researcher—analyzing vulnerability mechanisms, tracing logic, crafting trigger inputs, filtering noise, and delivering reproducible proofs of concept (PoCs).

In the recent CyberGym leaderboard, which contains 1,507 real‑world vulnerability tasks, Sangfor’s AI, powered by the fully domestic GLM‑5.2 model, completed 1,301 tasks, achieving an overall success rate of 86.3% and ranking in the global top four, the highest among Chinese teams.

The benchmark’s strict rules require AI not only to cause a crash but to establish a rigorous causal link between the observed anomaly and the target defect, ensuring that each result is evidence‑bound rather than a superficial answer.

Breaking down the results, the ARVO‑sourced tasks saw 1,193 completions (87.2% success) and the OSS‑Fuzz‑sourced tasks 108 completions (77.7% success), competing against teams from OpenAI, Anthropic, Meta and other leading organizations.

Beyond the headline score, Sangfor demonstrates a security‑oriented Agent system that converts model ability into concrete security outcomes. Vulnerability reproduction is a continual uncertainty‑reduction process: multiple plausible explanations, ambiguous code paths, and inputs that may not trigger the intended defect require repeated hypothesis, evidence collection, experimentation, and elimination of false leads.

A single model can generate local analyses or candidate PoCs, but long‑chain reasoning suffers from context drift, early misjudgments solidifying into defaults, and noisy logs. To address this, Sangfor’s system implements an "evidence governance" mechanism where hypotheses are freely generated but only accepted after rigorous evidence evaluation.

1. Expand hypotheses first, then let evidence decide their fate. The Agent Swarm keeps independent branches for vulnerability mechanism, code path, and trigger strategy, preventing early, unverified judgments from becoming collective defaults and avoiding anchoring bias.

2. Share evidence state, not full dialogue history. Branches synchronize only on hard evidence—confirmed code facts, runtime results, blocking conditions, and disproved paths—so subsequent agents inherit what has been ruled out rather than the entire conversation.

3. Dynamic orchestration to continuously converge uncertainty. A coordination layer evaluates evidence in real time, deciding which hypotheses remain viable, which branches need more proof, and which should be paused, merged, or terminated, ensuring each exploration meaningfully narrows the search space.

4. Separate conclusion generation from review. Candidate PoCs undergo an adversarial review assessing (1) stable reproducibility of the crash, (2) true correspondence to the target vulnerability, and (3) existence of a better explanation. Failed candidates are fed back as constraints for the next iteration.

This design extends the security field’s "default distrust" principle to the Agent system itself: internal consensus is insufficient without external, evidence‑backed validation.

The overall insight is that while the underlying model sets the upper bound of capability, the Agent system determines how much of that potential is realized as verifiable security results. The shift in competition rules now focuses on organizing model ability into stable, auditable security production rather than merely asking which model is stronger.

Using a consistent GLM‑5.2 base, Sangfor’s approach shows that when model reasoning is uniform, success hinges on managing parallel hypotheses, solidifying evidence, halting unproductive exploration, and rigorously discarding conclusions lacking causal support.