AutoResearch Myth Debunked: How Far Are Large Models From True Autonomous Research?

A comprehensive evaluation of 100 real-world research tasks across seven scientific domains reveals that current AI agents can execute experiments and generate reports but lack a metacognitive loop, causing them to recognize problems without correcting them, and exposing 45 distinct failure patterns that highlight a fundamental gap in autonomous scientific reasoning.

AI agentsAutoResearchFailure Taxonomy

0 likes · 10 min read

AutoResearch Myth Debunked: How Far Are Large Models From True Autonomous Research?

Architect

Mar 31, 2025 · Artificial Intelligence

A Comprehensive Study of Failure Modes in Large‑Language‑Model Based Multi‑Agent Systems

This paper presents a systematic investigation of failure patterns in LLM‑driven multi‑agent systems, introducing a 14‑type taxonomy (MASFT) derived from over 150 annotated dialogues, evaluating it with an LLM‑as‑a‑judge pipeline, and exploring modest intervention strategies while releasing all data and tools for future research.

AIAgenticFailure Taxonomy

0 likes · 29 min read

A Comprehensive Study of Failure Modes in Large‑Language‑Model Based Multi‑Agent Systems