Google's CLASHarness evaluates 15 leading LLMs on 600 multi-agent conflict tasks with 4K visual evidence, revealing top models achieve only 25–28% end-to-end pass rates despite 55–67% per-step scores, and that visual verification is critical for detecting teammate deception.
The author benchmarks Nanbeige4.3-33B against Qwen3.8-27B on an A100-80GB GPU, finding Nanbeige wins select benchmarks but fails to deliver its claimed 512K context on a single card due to high KV cache overhead from its Llama architecture, suffers from excessive chain-of-thought tokens that break coding tasks, and lacks vision capabilities, while Qwen's linear attention enables longer context and multimodal support.
ByteDance's TraceDance system automatically converts 252,557 real-world agent deployment trajectories into 107 behavioral benchmarks with 4,125 test instances, evaluating nine frontier models including Claude, GPT, and DeepSeek, which achieve only a 26.7% average pass rate on decision-point continuation tasks, revealing gaps between task completion and critical behavioral decisions.
Third-party Bug Hunt Bench results show DeepSeek Harness (DSH) with DeepSeek-V4.1-Flash-1017 surpasses Claude Code by 2-3 points across reasoning tiers, but consumes 82% more tokens and costs 82% more per task while fixing twice as many non-scored defects; the author recommends DSH for complex debugging and Claude Code for routine coding.
Lao Zhang evaluates DeepSeek-V4.1-Flash-1017, finding it China's closest model to global top-tier for coding and visual engineering with 10x cost savings, but weaker in writing and general knowledge versus GPT-6.1 and Gemini, while recommending Claude Code and DSH harness for optimal usage.
Anthropic released Claude Sonnet 5.5, which runs over 30% faster and costs up to 30% less than its predecessor, while matching Opus 5.5 on many benchmarks at half the price; it excels in coding, long-horizon tasks, and image understanding, with improved safety alignments and new distillation protections.
Researchers introduce Skill‑Entropy, a metric quantifying the difficulty of switching between reasoning skills in long‑horizon tasks, build the 558‑skill Skill²‑Bench, and show that Skill‑Entropy‑RL training dramatically improves cross‑skill performance of LLMs such as Qwen3, closing the gap observed in standard benchmarks.
BenchLoop is a CLI‑plus‑Web application that lets you reproducibly benchmark locally‑run LLMs across seven suites—including speed, tool‑calling, coding and agent tasks—while recording hardware details, scoring results with a weighted formula, and optionally publishing them to a public leaderboard.
DeepSeek V4 preview, released quietly on April 24, offers two models with 1 M token context and pricing 1/16 of Claude Opus, achieving near‑par performance on SWE‑bench and LiveCodeBench, while integration with Claude Code enables rapid project understanding, bug detection, refactoring, testing and documentation, saving days of work for under ¥6.
A detailed comparison of OpenAI's GPT-5.4, Anthropic's Claude, and Google's Gemini evaluates desktop agent performance, coding benchmarks, pricing, and use‑case suitability, revealing strengths, weaknesses, and cost considerations for developers and enterprises in 2026.
The article evaluates three Chinese open‑source large language models—GLM‑5, MiniMax‑M2.1 and Kimi‑K2.5—for use with the OpenClaw AI‑Agent gateway, comparing core specifications, programming and agent benchmarks, multimodal abilities, deployment costs, and scenario‑specific recommendations, while also sharing practical pitfalls.
