5×27B Qwen Agents Beat Fable 5 via Orchestration: Pipeline vs Single-Call Analysis
The article corrects a viral claim that five Qwen-72B agents beat Fable 5, revealing it was actually five Qwen3.8-27B models using a multi-agent orchestration (Memdoor team-solve) achieving 9.8 vs 8.0 on LiveCodeBench Hard, but clarifies this compares a 40-minute pipeline to a single call, with contamination concerns, and argues orchestration is a more cost-effective lever than model scale.
Correction: The Models Are 27B, Not 72B
The viral tweet from @QwenAIFans (Nov 2, 2026) claimed five Qwen3.8-72B agents scored 9.8 vs 8.0 against Fable 5 on LiveCodeBench Hard. The author identifies three reasons this is a typo:
Qwen 3.8 family's open-weight flagship for single-card/local deployment is the 27B variant (Apache 2.0, 262K context, tiny KV), while 72B is not a public tier in this generation.
The Memdoor GitHub repository (gureagodevo/memdoor, examples/workflows/team-solve) explicitly uses Qwen3.8-27B with a 128K response limit; all third-party reproductions (Clauding, AI Weekly, DevAIToolkit) record 27B.
Cost aligns: 50 runs cost $35.79 total, median $0.39 per run, 40 minutes each — consistent with consumer-grade small models, not a 72B cluster.
Therefore the actual setup is five Qwen3.8-27B instances .
Experimental Setup: Memdoor team-solve Workflow
Memdoor's team-solve is an agent pipeline CLI example with a counter-intuitive design:
Six roles (planner, two ideators, solver, manager, repair worker), each starting from a blank context cold start .
Agents do not share chat memory ; they communicate only via files on disk — planner writes a plan file, solver writes solution.py, manager reads test logs, repair worker patches.
Victory is decided by a script acting as judge (running sample tests), not human evaluation.
Benchmark: first 10 problems of LiveCodeBench Hard (Codeforces 1800+ difficulty), each attempted 5 times, scored on hidden tests.
The article includes a three-way comparison chart (Memdoor team-solve vs author's original harness vs Fable 5) and a diagram of the six roles' responsibilities and communication flow.
What "Beating Fable 5" Actually Means — Three Critical Layers
Layer 1: 9.8 matches the original harness, not a breakthrough. The workflow reproduces Lee (2026)'s GVS5H method; the original author's harness scored 9.6 on the same 10 problems. The 9.8 vs 9.6 difference is within noise — Memdoor proves it can reproduce and slightly exceed the pipeline, not that it's smarter than the method.
Layer 2: Pipeline vs single call, not model vs model. Fable 5's 8.0 comes from a single invocation per problem; Memdoor's 9.8 comes from six roles collaborating for 40 minutes. As Clauding notes, comparing a 40-minute multi-role collaboration to one frontier call is expected to win. The real comparison would be putting Fable 5 inside the same harness — until then, the 9.8 measures the harness's value, not model superiority.
Layer 3: Contamination is likely. The contest problems are from March–April 2025; the repository README states "model may have seen these problems during training, contamination cannot be ruled out." Combined with only 10 problems, the tweet's "grain of salt" disclaimer is necessary, not modest.
Objective conclusion: A structured multi-agent orchestration can squeeze 27B models to reach or exceed a frontier model's single-call score — but this is a victory for orchestration, not for small-model reasoning surpassing large models.
Why This Matters for Practitioners
1. Orchestration > scale — the 2026 lever to remember. Five roles communicating only via files, each cold-started, with a script as judge — this "structured debate + memory persistence" beats simple round-robin prompt chains. DevAIToolkit's reproduction noted the Critic role consistently caught hallucinated library calls that the solver would have accepted. When you're stuck with a single large model, the missing piece is often "someone to challenge it."
2. Extremely low cost, locally reproducible. 50 runs for $35.79 (median $0.39/run). Qwen3.8-27B quantized runs on consumer GPUs — a 64GB MacBook Pro can host multiple Ollama/mlx instances to replicate the pattern (slower, but functional).
3. The ceiling is higher. Today's comparison is "27B pipeline vs Fable 5 single call." If someone puts Fable 5 into the same team-solve harness, numbers will jump further — that's the true apples-to-apples. Until then, the takeaway: multi-agent orchestration is the most underestimated lever right now.
Replication Checklist
Stop throwing one prompt at one model. Decompose the task into planner / ideator / solver / verifier / repairer roles, each with independent context.
Force file-only communication to avoid shared long-context washing out objections (Memdoor's key mechanism is cross-turn memory persistence).
Attach a script as judge (run tests / samples) and use results to drive the next round, not model self-evaluation.
Model selection: locally use Qwen3.8-27B Q4 (one 64GB machine can spin multiple instances); cloud use open-weight APIs at pennies per call.
Expectation management: 10-problem sample + potential contamination means don't treat this as proof that small models have surpassed large ones, but as a strong signal that orchestration significantly boosts scores.
Final Thought
The tweet's most important message isn't the 9.8 vs 8.0 number, but the ignored "grain of salt" question: When five cheap small models, via division of labor and file communication, can approach an expensive frontier model's single-call level, should your next AI compute dollar go to a bigger model or a better orchestration?
Your next "reasoning machine" may not be the bigger one.
References: @QwenAIFans tweet (2026-11-02); Memdoor repo gureagodevo/memdoor (team-solve); Clauding / AI Weekly / DevAIToolkit reproductions; LiveCodeBench Hard public leaderboard. Note: tweet's "72B" verified as 27B typo.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Lao Guo's Learning Space
AI learning, discussion, and hands‑on practice with self‑reflection
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
