GPT-6 Wins B站 AI Arena, But Real-World Tests Show No Single Model Dominates

Introduction

On September 16, B站 launched its "AI Infinite Arena" and released the first large-model evaluation leaderboard. GPT-6 Astra took first place in 10 UP‑main evaluations, and domestic models occupied three of the top five spots. The community discussion quickly revealed a fundamental issue: the same GPT-6 performed very differently across UP‑main test categories — strong in code repair, occasional failures in game development, and even outperformed by some domestic models in life‑decision tasks. This inconsistency highlights the core dilemma of AI evaluation: what standard should measure a model's true capability?

1. What Does the B站 Arena Actually Test?

The arena's design is deliberately open: no fixed topics or dimensions. UP‑mains create tasks from their real workflows, letting hundreds of models compete on the same prompt. This contrasts sharply with traditional benchmarks (MMLU, HumanEval, SWE‑bench) which are standardized exams with fixed questions, answers, and scoring — akin to a national college entrance exam. The arena is a "real‑world trial": UP‑mains judge success based on whether the model actually completes the messy, multi‑step tasks they face daily.

2. What Did GPT‑6 Really "Crush"?

2.1 GPT‑6 Won, But the UP‑main Spoke a Hard Truth

The hardest code‑repair challenge, "Shanshan Lun Jian" ("Debate on the Shit Mountain"), asked models to fix legacy bugs where changing one line breaks three others. GPT‑6 scored well, yet the UP‑main "Token就是词元" cautioned: "GPT‑6 has indeed reached a new level, but whether it can remain consistently stable deserves further observation." This distinction between a one‑off high score and reliable delivery across hundreds of real tasks is crucial.

2.2 Why GPT‑6 Excels at Code Repair

GPT‑6 Astra's key upgrade is Computer Use — it can read screen pixels, control mouse and keyboard, and execute multi‑step tasks in real software. On OSWorld 2.0, Astra scored 72.6% versus GPT‑5.6 Sol's 65.7%, completing tasks in 40 minutes on average compared to Sol's 75 minutes — nearly twice as fast. In code repair this means GPT‑6 doesn't just point out errors; it enters the terminal, runs tests, reads stack traces, edits code, and re‑runs tests — a full closed loop.

2.3 But GPT‑6 Also Flips

In game‑development evaluations GPT‑6's performance was comparable to other top models. Creative design, gameplay logic, numerical balance, and player‑experience anticipation — tasks that standardized exams cannot capture — exposed its limits. Notably, GLM‑5.3 topped another code‑repair theme (legacy system testing), and Claude Fable 5.1 led in phased adversarial repair. The same broad "bug‑fix" direction produced different winners depending on the specific task design. This demonstrates that no single model "crushes" all real‑world scenarios; each has a distinct capability distribution. The author's own experience mirrors this: a model may excel at business‑logic coding but hallucinate when asked to infer product‑manager intent. The lesson: the best model is the one that matches your specific scenario, not the most expensive one.

3. The Core Contradiction

Many practitioners have seen a model top a leaderboard, integrated it eagerly, only to find it underperforms on the business scenario they care about most — sometimes worse than a lower‑scoring model.

This is a structural contradiction in current AI evaluation.

3.1 A Visual Summary

Traditional benchmarks test "do you know the right answer?" Real scenarios test "can you get the job done?" The required capabilities are fundamentally different.

3.2 Data Speaks

In May 2026, Shanghai AI Lab and universities released WildClawBench — 60 tasks simulating real work: crawling papers, auditing code repos, detecting API‑key leaks in Git history, extracting structured data from meeting recordings to generate professional brochures. The best model, Claude Opus 4.6, scored only 51.6%. Even the top model completes barely half of true end‑to‑end tasks. Another benchmark, BeyondSWE from Renmin University's High‑Level AI Institute, tests cross‑repo retrieval, domain knowledge, dependency upgrades, and building systems from scratch. Top models' pass rates plummeted below 45%, revealing a lack of genuine engineering thinking.

3.3 GPT‑6's Own Numbers Tell the Story

GPT‑6's widely cited 99.9% on ARC‑AGI‑3 comes with a critical caveat. The ARC Prize official standard harness yielded 62.7%; the Provider Adapter harness (which allows the model to carry notes) gave 99.9%. Both are called SOTA, but the difference is whether the model can "remember" more. More importantly, on a composite of 9 knowledge, coding, tool‑use, and long‑context tests, GPT‑6 Astra's General Intelligence Index is 61 — tied with GPT‑5.6 Sol and 5 points below Claude Fable 5.1's 66. In short, GPT‑6's "intelligence" hasn't surpassed its predecessor; its breakthrough is in "hands‑on" execution ability.

4. K3's Elimination: A Case Worth Reflecting On

Kimi K3, a 2.8 trillion ‑parameter open‑source model with 1M context that topped frontend leaderboards in July, did not appear on the arena's first leaderboard. Forum users questioned this; the top reply: "Scores and actual scenarios are not the same thing." K3 remains strong at frontend UI generation, but real‑world evaluation exposes its capability boundaries — it excels at some task types and lags at others. Traditional benchmarks often only test what a model is best at.

5. How Should We View GPT‑6's "Championship"?

GPT‑6 Astra is undeniably one of the strongest programming and agent models today. Its arena win, OSWorld 2.0 lead ( 72.6% vs 65.7%), Codeforces rating 3206, and BenchCAD score 95.9% are solid achievements. Yet "crushing victory" obscures a more important fact: AI capability is multidimensional. GPT‑6 leads in agent execution and code repair; GLM‑5.3 in legacy system testing; Claude Fable 5.1 in complex debugging and reasoning; Kimi K3 in frontend UI generation. This is not "who replaces whom" but "each has its strengths."

5.1 Where Is GPT‑6's Real Breakthrough?

Not "more knowledge" but from "talking" to "doing." OpenAI's positioning — "Anything you can do on a computer, Astra can do for you" — is a directional statement, not marketing fluff. Previous models answered questions or told you how to do something; you executed. GPT‑6 directly does it: reads the screen, clicks, types, runs tests, reads errors, rewrites code. It delivers a closed loop, not just an output. On Terminal‑Bench 4.0, GPT‑6 scores 57.9% versus Sol's 37.3% — a >50% improvement in programming execution.

Conclusion

Returning to the Zhihu question: "GPT‑6 crushed the UP‑main crowd test — is it now the strongest model?" The answer: It is among the strongest in agent execution and code repair, but "strongest model" is a flawed notion because "strongest" depends on the ruler you use. If you measure "can it get the job done" — GPT‑6 is strong. If you measure "general intelligence" — it's on par with the previous generation. If you measure "leads in every real scenario" — the answer is clearly no.

A model topping one leaderboard does not mean it outperforms in all real scenarios. Strong crowd‑test performance doesn't guarantee higher comprehensive capability — it may simply align with the tasks those evaluators designed.

AI evaluation is shifting from "exams" to "real‑world validation." B站's arena provides a new window beyond traditional benchmarks. In 2026 the evaluation landscape is undergoing a deep transition: from score‑chasing to scenario verification, from single‑dimension testing to multi‑dimensional assessment of safety, trustworthiness, and robustness.

Can this window replace traditional benchmarks? Not replacement, but complement. Official benchmarks enable horizontal comparison; real‑scenario tests reveal actual user experience. Together they give a complete picture of a model.

Don't just watch scores, and don't just watch one UP‑main's video. Watch several, run several real tasks, and the answer will emerge naturally.