Skip to content
Tagged articles

Agent Evaluation

47 articles · Page 1 of 1
Architect
Architect
Nov 16, 2026 · Artificial Intelligence

Anthropic's Daily Brief: Designing Agent Automations That Reduce Human Verification Work

The article deconstructs Anthropic's Daily Brief example to show how effective agent automation requires cursor-based progress tracking, pre-send verification via ledgers, idempotent delivery confirmation, strict permission boundaries, budget guards, and evaluation against human-labeled ground truth to ensure automation truly reduces workload rather than shifting it to verification.

Agent AutomationAgent EvaluationAnthropic
0 likes · 38 min read
Anthropic's Daily Brief: Designing Agent Automations That Reduce Human Verification Work
Qunhe Technology Quality Tech
Qunhe Technology Quality Tech
Nov 11, 2026 · Artificial Intelligence

Evaluating AI Agents: From Core Concepts to Reliable Business Decisions

This article explains AI Agent fundamentals using a personal assistant analogy, then details how to design evaluation cases, interpret results across quality, stability, and cost dimensions, compare versions, and use evaluation data to make informed release decisions, illustrated with a home renovation Agent case study.

AI agentAgent EvaluationEvaluation Design
0 likes · 21 min read
Evaluating AI Agents: From Core Concepts to Reliable Business Decisions
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Nov 10, 2026 · Artificial Intelligence

WorkflowArena Benchmark: Top LLMs Achieve Only 36.95% in Multi-Agent Orchestration

Einsia AI's WorkflowArena benchmark evaluates 9 frontier models across 145 long-horizon tasks in 8 domains, revealing that even the best system (Claude Opus 5.5) achieves only 36.95% full solve rate, exposing multi-agent orchestration as a critical weakness, with distinct orchestration styles between Claude and GPT models, and showing that strong orchestration can significantly boost weaker executors.

AI agentsAgent EvaluationClaude Opus 5.5
0 likes · 17 min read
WorkflowArena Benchmark: Top LLMs Achieve Only 36.95% in Multi-Agent Orchestration
IT Services Circle
IT Services Circle
Nov 10, 2026 · Artificial Intelligence

10 High-Frequency AI Agent Interview Questions: Technical Deep Dive with Code & Diagrams

This comprehensive guide covers 10 essential AI Agent interview questions, detailing core concepts like Agent vs LLM, Workflow vs Agent architectures, reasoning paradigms (ReAct, CoT, Plan-and-Execute, Self-Refine), Function Calling, MCP, error handling, memory systems, context management, RAG pipelines, and evaluation methodologies with code examples and diagrams.

AI agentAgent EvaluationFunction Calling
0 likes · 39 min read
10 High-Frequency AI Agent Interview Questions: Technical Deep Dive with Code & Diagrams
Tech Freedom Circle
Tech Freedom Circle
Nov 6, 2026 · Artificial Intelligence

Langfuse Three-Level Agent Evaluation: Outcome, Trajectory & Step Framework Principles & Implementation

This article details Langfuse's three-level evaluation framework for AI agents—Outcome, Trajectory, and Step—explaining how its Trace, Evaluator, Dataset, and Experiment components enable decoupled observability and evaluation, with concrete implementation patterns for code-based rules, LLM-as-a-Judge, and human annotation to build reliable CI/CD gates and production monitoring loops.

AI TestingAgent EvaluationCI/CD Gates
0 likes · 33 min read
Langfuse Three-Level Agent Evaluation: Outcome, Trajectory & Step Framework Principles & Implementation
DataFunTalk
DataFunTalk
Nov 4, 2026 · Artificial Intelligence

How 1688 Built a Self-Correcting Agent Evaluation System with Dual-Track Testing

Alibaba's 1683 shares its production-grade Agent evaluation framework combining offline golden sets with online log sampling, using Task×Trial×Trace×Outcome decomposition, executable Rubrics with deterministic gates, LLM Judges constrained by evidence citations and rule-based scoring, and a closed loop that routes failures to targeted fixes and regression testing.

AI agentAgent EvaluationDual-Track Evaluation
0 likes · 20 min read
How 1688 Built a Self-Correcting Agent Evaluation System with Dual-Track Testing
PaperAgent
PaperAgent
Oct 30, 2026 · Artificial Intelligence

FDE-Gym Benchmark Exposes AI Delivery Gap: Top Models Solve Only 14 of 95 Tasks

Tsinghua, Microsoft, and collaborators introduce FDE-Gym, a reproducible benchmark for AI delivery agents across 20 customer scenarios and 95 tasks, revealing that even Claude Opus 5.5 solves only 14 tasks at the most realistic level, with the core bottleneck being discovery of implicit constraints and translation of facts into deployed systems.

AI delivery agentsAgent EvaluationBenchmark
0 likes · 13 min read
FDE-Gym Benchmark Exposes AI Delivery Gap: Top Models Solve Only 14 of 95 Tasks
AI Digital Ideal
AI Digital Ideal
Oct 29, 2026 · Artificial Intelligence

AI Agent Metacognition & Evaluation: Confidence Scoring, Benchmarks & Production-Grade Eval Frameworks

This lesson teaches how to implement metacognition in AI agents — enabling self-assessment via confidence scores, self-consistency, multi-model voting, and internal knowledge checks — and covers a complete three-layer evaluation framework (task, trajectory, experience) with runnable code for production-grade evaluation harnesses, key metrics like pass@k vs pass^k, and major benchmarks including HumanEval, SWE-bench, GAIA, and WebArena.

AI agentsAgent EvaluationBenchmarks
0 likes · 22 min read
AI Agent Metacognition & Evaluation: Confidence Scoring, Benchmarks & Production-Grade Eval Frameworks
Tech Freedom Circle
Tech Freedom Circle
Oct 29, 2026 · Artificial Intelligence

6 Agent Baseline Metrics & 4 RAG Metrics: Definitions, Formulas, and Principles Explained

This article systematically breaks down six industrial-grade Agent evaluation metrics (PlanQuality, PlanAdherence, ToolCorrectness, ArgumentCorrectness, TaskCompletion, StepEfficiency) and four RAG baseline metrics (Faithfulness, Answer Relevancy, Context Precision, Context Recall), detailing their definitions, calculation formulas, underlying principles, and key differences in evaluation scope, failure modes, granularity, and judge alignment targets.

AI-evaluationAgent EvaluationDeepEval
0 likes · 22 min read
6 Agent Baseline Metrics & 4 RAG Metrics: Definitions, Formulas, and Principles Explained
Architect
Architect
Oct 28, 2026 · Artificial Intelligence

Anthropic: Over-Splitting Sub-Agents Adds Latency — Four Consolidation Patterns

Anthropic finds that splitting AI agents by business function often increases latency due to handoff overhead; they recommend consolidating sub-agents into a main loop using four patterns — merge, fold, as-tool, keep — with Skills for business rules, tools for actions, and task-level evaluation to catch regressions.

AI agentsAgent EvaluationAnthropic
0 likes · 24 min read
Anthropic: Over-Splitting Sub-Agents Adds Latency — Four Consolidation Patterns
Alibaba Cloud Native
Alibaba Cloud Native
Oct 27, 2026 · Cloud Native

Building Trustworthy AIOps Evaluation Datasets: STAROps Root Cause Analysis Practices

Alibaba Cloud details its methodology for constructing and maintaining high-quality evaluation datasets for the STAROps AIOps agent, covering fault injection, signal capture, evidence extraction, quality checks, environment construction, and continuous iteration to ensure reliable root cause analysis assessments.

AIOpsAgent EvaluationRoot Cause Analysis
0 likes · 42 min read
Building Trustworthy AIOps Evaluation Datasets: STAROps Root Cause Analysis Practices
DataFunTalk
DataFunTalk
Oct 25, 2026 · Artificial Intelligence

EvoMemBench: Why AI Agent Memory Must Evolve, Not Just Remember

The article introduces EvoMemBench, a benchmark evaluating AI agent memory across four dimensions — task-internal knowledge, task-internal execution, cross-task knowledge, and cross-task execution — revealing five key findings: memory isn't always better than raw context, helps more on hard tasks, is critical for long-horizon execution, retrieval does not equal revision, and no single memory type fits all tasks. It concludes with five engineering principles for building effective memory systems.

AI agent memoryAgent EvaluationEvoMemBench
0 likes · 20 min read
EvoMemBench: Why AI Agent Memory Must Evolve, Not Just Remember
SmartApe AI
SmartApe AI
Oct 24, 2026 · Artificial Intelligence

ByteDance TraceDance: 250K Real Trajectories Expose Frontier Model Behavioral Failures

ByteDance's TraceDance system automatically converts 252,557 real-world agent deployment trajectories into 107 behavioral benchmarks with 4,125 test instances, evaluating nine frontier models including Claude, GPT, and DeepSeek, which achieve only a 26.7% average pass rate on decision-point continuation tasks, revealing gaps between task completion and critical behavioral decisions.

Agent EvaluationBehavioral BenchmarksByteDance
0 likes · 17 min read
ByteDance TraceDance: 250K Real Trajectories Expose Frontier Model Behavioral Failures
Old Zhang's AI Learning
Old Zhang's AI Learning
Oct 23, 2026 · Artificial Intelligence

Anthropic Warns: Ditch Subagents for a Single Loop with Skills

Anthropic argues that multi-subagent architectures often mirror organizational charts rather than technical needs, showing single-loop agents with progressive skill disclosure outperform complex pipelines in quality, latency, and cost, and provides a migration framework.

AI agentsAgent EvaluationAnthropic
0 likes · 22 min read
Anthropic Warns: Ditch Subagents for a Single Loop with Skills
Data Bricklaying Diary
Data Bricklaying Diary
Oct 22, 2026 · Big Data

Continuous Delivery for High-Quality Datasets: Dataset CI & Eval CI Pipelines

The article details a continuous delivery pipeline for high-quality datasets, combining Dataset CI for contract validation and Eval CI for task regression, with automated quality gates, manual approvals, canary releases, and full-baseline rollback, using a policy knowledge base as a running example.

Agent EvaluationContinuous DeliveryData Governance
0 likes · 29 min read
Continuous Delivery for High-Quality Datasets: Dataset CI & Eval CI Pipelines
Data Bricklaying Diary
Data Bricklaying Diary
Oct 17, 2026 · Artificial Intelligence

Agent Calls Succeed But Answers Fail: Tracing Observability to Quality Evaluation

The article explains how successful Agent execution traces can still produce wrong answers, using a procurement case where an Agent cited an outdated contract attachment. It details a systematic approach: trace errors through observability across indexing, retrieval, context assembly, and generation; convert feedback into reproducible test samples with versioned inputs and expected behaviors; evaluate using rule-based checks, model-assisted judging, and human review; separate critical error categories instead of relying on a single score; maintain regression suites with both failure and success cases; and rerun tests in isolated environments while recording versions to distinguish new results from original incidents.

Agent EvaluationRegression TestingRetrieval-Augmented Generation
0 likes · 15 min read
Agent Calls Succeed But Answers Fail: Tracing Observability to Quality Evaluation
DataFunSummit
DataFunSummit
Oct 16, 2026 · Artificial Intelligence

AWS Strands Harness Cuts Agent Costs 77% vs Claude Code by Optimizing Context, Not Model

AWS open-sourced Strands Harness, a pre-assembled agent runtime that decouples the model from the execution system; benchmarks show using the same Claude Fable 5 model, Strands achieves 77% lower cost and higher accuracy than Claude Code by optimizing context management, prompt caching, and layered memory, proving harness design critically impacts agent performance.

AWSAgent EvaluationAgent Runtime
0 likes · 18 min read
AWS Strands Harness Cuts Agent Costs 77% vs Claude Code by Optimizing Context, Not Model
Wu Shixiong's Large Model Academy
Wu Shixiong's Large Model Academy
Oct 14, 2026 · Artificial Intelligence

Verifying Agent Task Completion: Three Success Layers, Acceptance Criteria, and Timeout Recovery

The article explains why a tool's success response doesn't guarantee an Agent task is done, detailing three verification layers (call, execution, task), upfront acceptance criteria, timeout recovery via state reconciliation, and evaluation metrics that separate run completion from true task acceptance.

AI agentsAcceptance TestingAgent Evaluation
0 likes · 18 min read
Verifying Agent Task Completion: Three Success Layers, Acceptance Criteria, and Timeout Recovery
Wu Shixiong's Large Model Academy
Wu Shixiong's Large Model Academy
Oct 9, 2026 · Artificial Intelligence

Why Your 93% Agent Success Rate Collapses Under Scrutiny: The Evaluation Gap Between Saying and Doing

The article details a rigorous agent evaluation framework for after-sales operations using GRPO, revealing why a 93% overall success rate masks critical failures in specific scenarios and safety guardrails, and how to structure evaluations to withstand interviewer scrutiny.

Agent EvaluationBucket AnalysisGRPO
0 likes · 18 min read
Why Your 93% Agent Success Rate Collapses Under Scrutiny: The Evaluation Gap Between Saying and Doing
AI2ML AI to Machine Learning
AI2ML AI to Machine Learning
Sep 29, 2026 · Artificial Intelligence

Agent Kung-fu: 10 Layers of Control Transfer for Production-Ready Agents

This article presents a 10-level framework for building production-ready AI agents, centered on six critical questions about action control, capability access, conflict resolution, failure handling, knowledge reuse, and causal reasoning, plus four engineering principles emphasizing code-enforced permissions, evaluator-driven loops, identification before optimization, and falsifiable innovation.

AI agentsAgent EvaluationProduction Readiness
0 likes · 5 min read
Agent Kung-fu: 10 Layers of Control Transfer for Production-Ready Agents
DataFunSummit
DataFunSummit
Sep 28, 2026 · Artificial Intelligence

10 Financial Firms Share AI Agent Strategies for 98.5% Auto-Review, 0.003% Fraud

This article analyzes how 10 leading financial institutions implement AI agents in low-tolerance scenarios, detailing their approaches to data ontology, semantic layers, multi-agent architectures, risk control, and evaluation frameworks, achieving metrics like 98.5% automated review rates and 0.003% fraud rates while ensuring auditability and regulatory compliance.

AI agentsAgent EvaluationApache Ossie
0 likes · 43 min read
10 Financial Firms Share AI Agent Strategies for 98.5% Auto-Review, 0.003% Fraud
Machine Heart
Machine Heart
Sep 26, 2026 · Artificial Intelligence

Right Answer, Wrong Reason: LexAgentHallu Benchmarks Hidden Hallucinations in Legal AI Agents

HKUST researchers introduce LexAgentHallu, a hierarchical benchmark that evaluates hallucinations in legal AI agents by tracing entire reasoning trajectories, revealing that even top-performing systems exhibit hallucinations in 89% of execution traces and 68% of correct answers contain flawed reasoning.

Agent EvaluationBenchmarkHKUST
0 likes · 14 min read
Right Answer, Wrong Reason: LexAgentHallu Benchmarks Hidden Hallucinations in Legal AI Agents
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Sep 21, 2026 · Artificial Intelligence

LongDS v1.1 Benchmark Released: GPT-6 Astra Tops Long-Horizon Data Analysis Lite Leaderboard

Zhejiang University and Ant Group release LongDS v1.1, a benchmark for long-horizon multi-turn data analysis agents built from real Kaggle workflows; the Lite subset of 24 tasks and 777 turns shows GPT-6 Astra leading at 78.17, with detailed error analysis revealing cascading state errors as the primary failure mode.

Agent EvaluationBenchmarkEMNLP 2026
0 likes · 24 min read
LongDS v1.1 Benchmark Released: GPT-6 Astra Tops Long-Horizon Data Analysis Lite Leaderboard
Su San Talks Tech
Su San Talks Tech
Sep 21, 2026 · Artificial Intelligence

AI Agent Interview Deep Dive: 10 Critical Questions from Architecture to Evaluation

This comprehensive guide covers 10 essential AI Agent interview topics, including Agent vs LLM differences, Workflow vs Agent selection, reasoning paradigms, Function Calling, MCP, error handling, memory management, context optimization, RAG pipelines, and evaluation metrics, with code examples and architectural diagrams.

AI agentAgent EvaluationFunction Calling
0 likes · 43 min read
AI Agent Interview Deep Dive: 10 Critical Questions from Architecture to Evaluation
DataFunSummit
DataFunSummit
Sep 18, 2026 · Artificial Intelligence

Palantir's AI FDE Automates Forward Deployment While China Still Recruits Human FDEs

Palantir's AI Forward Deployed Engineer (FDE) now automates execution tasks like data integration and ontology management within Foundry, while AIP Evolve enables agents to self-optimize models and prompts via eval-driven loops; Chinese enterprises similarly adopt AI FDE to parallelize delivery workflows, but human judgment remains essential for defining context, correctness, and error boundaries.

AI-FDEAIP EvalsAIP Evolve
0 likes · 21 min read
Palantir's AI FDE Automates Forward Deployment While China Still Recruits Human FDEs
Machine Heart
Machine Heart
Sep 16, 2026 · Artificial Intelligence

Harness Evolution vs. Test-Time Scaling: Simple Retries Outperform Complex Self-Improvement

A study from AI2 and University of Washington finds that complex Harness Evolution for AI agents fails to consistently outperform simple test-time scaling methods like parallel sampling under equal compute budgets, and improvements rarely transfer to unseen tasks, questioning whether observed gains stem from genuine self-improvement or just extra attempts.

AI agentsAgent EvaluationHarness Evolution
0 likes · 14 min read
Harness Evolution vs. Test-Time Scaling: Simple Retries Outperform Complex Self-Improvement
dbaplus Community
dbaplus Community
Sep 15, 2026 · Artificial Intelligence

Meituan's Agent Evaluation System: From Scoring to Infrastructure Capability

Meituan's Turing team shares a comprehensive framework for evaluating AI agents, covering multi-layered assessment (result, process, efficiency, risk), human-machine alignment via binary rubrics, seed test sets, expert knowledge integration, and the shift toward infrastructure for long-horizon agents with task-based evaluation harnesses.

Agent EvaluationEvaluation InfrastructureHuman-Machine Alignment
0 likes · 33 min read
Meituan's Agent Evaluation System: From Scoring to Infrastructure Capability
dbaplus Community
dbaplus Community
Sep 9, 2026 · Artificial Intelligence

Production-Grade Enterprise Agents: Unifying Harness, Skills & Virtual File Systems

This article details a production-grade architecture for enterprise AI agents, combining a unified harness for execution control, federated skills for domain expertise, and a virtual file system for long-task context management, drawing on Stripe's Kai platform and Deep Agents framework to address governance, security, and scalability challenges.

AI governanceAgent EvaluationAgent Harness
0 likes · 37 min read
Production-Grade Enterprise Agents: Unifying Harness, Skills & Virtual File Systems
Alibaba Cloud Native
Alibaba Cloud Native
Sep 4, 2026 · Artificial Intelligence

Close the Loop Finale: Agent Evaluation, SkillOps & LangChain Engineering Practices

The Shanghai finale of the Agent Observability and Optimization Close the Loop tour featured three technical sessions on building verifiable Agent optimization loops, managing Skills as strategic assets with SkillOps, and applying LangChain's Agent Engineering lifecycle from prototype to production, plus a hands-on workshop using Qoder and PawBench to demonstrate end-to-end evaluation.

Agent EvaluationAgentLoopLangChain
0 likes · 9 min read
Close the Loop Finale: Agent Evaluation, SkillOps & LangChain Engineering Practices
Chengwu Tech Stack
Chengwu Tech Stack
Aug 31, 2026 · Artificial Intelligence

Autonomous vs. Engineered Agents: Balancing Flexibility and Control in Production

This article distinguishes autonomous agents that dynamically plan actions from engineered agents that embed decision-making within verifiable, auditable workflows, providing criteria for choosing each approach, a three-layer production architecture, and a checklist for determining the right level of autonomy based on risk, reversibility, and audit requirements.

AI agentsAgent Evaluationagent architecture
0 likes · 16 min read
Autonomous vs. Engineered Agents: Balancing Flexibility and Control in Production
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Aug 26, 2026 · Artificial Intelligence

Can AI Do Independent Research? ASI‑Bench Measures Scientific Autonomy

ASI‑Bench, developed by Tsinghua and leading institutions, is a benchmark that evaluates AI’s scientific autonomy by progressively reducing method guidance across four levels, revealing that current models lose up to half their scientific score without detailed instructions, highlighting the gap to true independent research.

AI autonomyASI-BenchAgent Evaluation
0 likes · 14 min read
Can AI Do Independent Research? ASI‑Bench Measures Scientific Autonomy
Huolala Tech
Huolala Tech
Aug 12, 2026 · Artificial Intelligence

Redefining Agent Evaluation: An Offline LLM-as-a-Judge Framework and Practice

The article presents a quantitative offline evaluation system for AI agents in real outbound-call scenarios, combining reference‑based scoring with pairwise GSB methods, addressing regression and optimization, mitigating systematic bias through judge model selection and majority‑vote adjustments, and delivering an objective, high‑efficiency benchmark.

Agent EvaluationGSBLLM-as-a-Judge
0 likes · 16 min read
Redefining Agent Evaluation: An Offline LLM-as-a-Judge Framework and Practice
Meituan Technology Team
Meituan Technology Team
Aug 6, 2026 · Artificial Intelligence

A Deep Dive into Agent Evaluation: From Basics to Advanced Practices

This article explains why evaluating AI agents requires more than answer correctness, outlines a four‑layer evaluation framework (result, process, efficiency, risk), compares short‑ and long‑horizon agents, and presents a practical methodology that combines objective and subjective metrics, rubric binary‑ization, case management, and infrastructure requirements for scalable, repeatable agent testing.

AI agentAgent EvaluationEvaluation Framework
0 likes · 25 min read
A Deep Dive into Agent Evaluation: From Basics to Advanced Practices
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jun 24, 2026 · Artificial Intelligence

Can Agents Truly Self‑Evolve? GDPevo Benchmark That No Agent Can Cheat

The article introduces GDPevo, the first open‑source benchmark that quantifies self‑evolution in agents by generating 120 real‑world enterprise tasks, using rule‑hybrid question creation and deterministic scoring, and shows that self‑evolving agents improve accuracy by 17‑22% while reducing token consumption.

AI benchmarkAgent EvaluationContinual Learning
0 likes · 12 min read
Can Agents Truly Self‑Evolve? GDPevo Benchmark That No Agent Can Cheat
AI Engineering
AI Engineering
Jun 1, 2026 · Artificial Intelligence

Why Do Most Agent Projects Fail Before Launch? LangChain’s Solution

The article explains why many AI Agent projects collapse before production due to non‑determinism, error propagation, and creative solutions, and presents LangChain’s Deep Agent evaluation framework—integrated with LangSmith, AWS Bedrock, and Pytest—to provide a reproducible, end‑to‑end testing and monitoring process.

AWS BedrockAgent EvaluationDeep Agent
0 likes · 9 min read
Why Do Most Agent Projects Fail Before Launch? LangChain’s Solution
PaperAgent
PaperAgent
May 25, 2026 · Artificial Intelligence

DeepSeek’s Harness: How Agent Harness Engineering Is Shaping the Next LLM Agent Era

The article surveys DeepSeek’s Harness initiative, presenting the Binding‑Constraint Thesis, three‑stage evolution from prompt to harness engineering, the ETCLOVG seven‑layer architecture, and concrete benchmark evidence that harness‑only improvements far outweigh model upgrades, while detailing security, observability, and governance considerations for reliable LLM agents.

AI ArchitectureAgent EvaluationAgent Harness Engineering
0 likes · 12 min read
DeepSeek’s Harness: How Agent Harness Engineering Is Shaping the Next LLM Agent Era
ITPUB
ITPUB
May 16, 2026 · Artificial Intelligence

Managing AI‑Generated Code with an Agent‑Based Evaluation Framework: Lessons from Refactoring 310 K Lines

When over 90% of a codebase is produced by AI, the authors show how a unified "people‑align → human‑machine‑align" approach, driven by evaluation agents, transforms technical debt into incremental business work, enabling continuous refactoring, AI‑friendly standards, and a sustainable engineering environment.

AI codingAI governanceAgent Evaluation
0 likes · 21 min read
Managing AI‑Generated Code with an Agent‑Based Evaluation Framework: Lessons from Refactoring 310 K Lines
Meituan Technology Team
Meituan Technology Team
May 7, 2026 · R&D Management

Managing AI‑Generated Code with Agent‑Based Evaluation: Refactoring 310K Lines of Code

When over 90% of a codebase is produced by AI, system quality hinges on constraining AI rather than speed, and this article details how a team used an agent‑based evaluation framework, unified standards, and incremental refactoring to turn 310,000 lines of AI‑written code into a maintainable, low‑debt system.

AI codingAI governanceAgent Evaluation
0 likes · 21 min read
Managing AI‑Generated Code with Agent‑Based Evaluation: Refactoring 310K Lines of Code
AntData
AntData
Apr 28, 2026 · Artificial Intelligence

Iterative Agent Evaluation Skill: Automating Bad‑Case Diagnosis with AI Pre‑Annotation

The article presents an end‑to‑end, eight‑phase automated evaluation pipeline for large‑model agents that replaces manual bad‑case inspection with AI‑assisted pre‑annotation, cutting analysis time from a full‑day to about 30 minutes and achieving over 90 % efficiency gain while enabling iterative knowledge‑base refinement.

AI pre‑annotationAgent EvaluationAutomated Pipeline
0 likes · 20 min read
Iterative Agent Evaluation Skill: Automating Bad‑Case Diagnosis with AI Pre‑Annotation
Alibaba Cloud Developer
Alibaba Cloud Developer
Mar 27, 2026 · Artificial Intelligence

How OpenClaw Empowers a Self‑Evolving Bank Manager Assistant

This article details a three‑day deep dive into OpenClaw, demonstrating how a self‑iterating AI assistant for bank relationship managers can be built, validated, and refined through autonomous agent communication, scheduled tasks, and memory‑driven reflection.

AI agentsAgent EvaluationMemory Architecture
0 likes · 20 min read
How OpenClaw Empowers a Self‑Evolving Bank Manager Assistant
PaperAgent
PaperAgent
Dec 23, 2025 · Artificial Intelligence

CATArena: A Competitive Benchmark That Turns Agent Scoring into Evolutionary Learning

CATArena introduces a tournament‑style evaluation framework where AI agents iteratively code, compete, and improve across classic board games, using three‑dimensional quantitative scores to measure strategy programming, global learning, and generalization, and reveals how different LLM‑based agents learn and adapt over multiple rounds.

AI benchmarkAgent EvaluationCATArena
0 likes · 8 min read
CATArena: A Competitive Benchmark That Turns Agent Scoring into Evolutionary Learning
Amazon Cloud Developers
Amazon Cloud Developers
Dec 23, 2025 · Artificial Intelligence

Evaluating Agent Quality: A Practical Guide for Agentic AI

This article explains why evaluating AI agents is essential, outlines a multi‑dimensional metric system covering performance, safety, cost and bias, describes common evaluation frameworks such as AgentBoard, AgentBench and τ‑bench, and provides step‑by‑step instructions, example datasets and code for building a robust agent assessment pipeline.

AI agentsAgent EvaluationLLM
0 likes · 35 min read
Evaluating Agent Quality: A Practical Guide for Agentic AI
DataFunTalk
DataFunTalk
Jul 14, 2025 · Artificial Intelligence

Can Kimi K2 Beat Claude and Gemini in Coding and Agent Tasks?

This in‑depth review examines Kimi K2’s new focus on agent and coding abilities, comparing its performance on 3D HTML generation, code generation, and real‑world agent tasks against Claude 4 and Gemini 2.5, while also evaluating cost, openness, and practical usability for developers.

AI codingAgent EvaluationKimi K2
0 likes · 15 min read
Can Kimi K2 Beat Claude and Gemini in Coding and Agent Tasks?