Xiaomi's TRACE Method Wins IJCAI-ECAI 2026 CAR-bench Challenge with 70% Stable Pass Rate

Competition Background

The IJCAI-ECAI 2026 conference, held in Bremen, Germany, hosted the CAR-bench Challenge, a competition focused on task execution and reliability of in-car voice assistants in real-world scenarios. The challenge was backed by automotive industry players, foundation model vendors, AI compute platforms, and universities, attracting teams from BMW Group, Northeastern University, the Alan Turing Institute, and the University of Warwick.

The competition environment simulated a complete vehicle cabin with 58 real callable tools covering navigation, windows, climate control, charging, calendar, and more. Two tracks were offered: a restricted track with specified models and limited compute, and an open track allowing free choice of models and system architectures to explore frontier model capabilities and real-world deployment stability. The core evaluation metric required each task to be repeated three times, with the stable pass rate (all three runs successful) as the primary measure of reliability.

Challenge Design: Three Test Categories

The benchmark designed three categories of test cases to probe different failure modes of in-car agents:

Basic tasks: Clear instructions; tests whether the agent can execute step-by-step correctly.

"Knowingly impossible" tasks: Required tools are deliberately removed; tests whether the agent fabricates tool calls and fake results or honestly admits inability.

"Ambiguous instruction" tasks: Vague user commands; tests whether the agent guesses blindly or actively clarifies user intent before acting.

These three categories directly correspond to the capabilities needed for real-world deployment: task completion, self-awareness of capability boundaries, and proactive disambiguation under uncertainty.

TRACE Method: Trajectory-based Agent Continuous Evolution

The Darwin Agent team proposed TRACE (Trajectory-based Agent Continuous Evolution), a method that improves stability without retraining or modifying the underlying model. TRACE operates in three steps:

Step 1: Decompose Experience into Reusable Atomic Actions

Traditional approaches summarize experience into broad task-level guides (e.g., "what to do when the cabin feels stuffy"), which are hard to transfer across scenarios. TRACE instead breaks down task goals into concrete, reusable operations such as "open window" or "turn on fan." These atomic actions can be reused across navigation, climate, media, and other tasks, increasing composability and transferability.

Step 2: Compare Correct and Incorrect Execution Traces to Locate Failure Causes

Judging only by final task success or failure often masks the exact step where things went wrong. TRACE compares execution traces of successful and failed runs for the same task type, pinpointing specific failure causes: executing with insufficient information, omitting critical rules, or hallucinating non-existent capabilities. These diagnosed failure patterns are then stored in an experience base for continuous refinement.

Step 3: Retrieve Experience On-Demand Based on Dialogue Context

Instead of stuffing all accumulated experience into the model context, TRACE selects only the experiences and rules relevant to the current dialogue turn. As the user's request evolves, the retrieved experience set changes dynamically. This reduces compute overhead and keeps the agent's behavior tightly aligned with the immediate situation.

Results

On the open track, TRACE achieved an at-least-once success rate of approximately 83% across three repeated runs. More importantly, the stable pass rate (all three runs successful) reached 70%, a 20-percentage-point improvement over the official baseline of 50%. This stable pass rate secured first place in the open track and earned both the Rank Award and the Innovation Award.

Implications for Real-World Deployment

The competition validated three core capabilities essential for production in-car assistants: stable task completion, honest admission of inability, and proactive clarification under ambiguity. Real-world examples illustrate the stakes: when a user says "it's stuffy," the agent must check vehicle speed, outside temperature, and rain status before deciding between opening a window or activating climate control; when a user says "navigate to the office," the agent must disambiguate among multiple stored office addresses rather than picking arbitrarily.

TRACE's experience-accumulation loop — distilling successful runs into "guidance experience" and failed runs into "error experience," then retrieving them contextually — provides a practical path to improving reliability without model retraining. The method's principles are relevant to Xiaomi's intelligent cockpit, Xiao Ai voice assistant, and broader "human-car-home" ecosystem, offering a new direction for building dependable agents in real-world settings.