AdaptEvo: From Rule Matching to Evidence-Based Contextual Decisions in AI Agents

Background: Rules Require Contextual Interpretation

Example: An image containing a wound – whether it violates policy depends on severity, presentation, and context (e.g., medical education exemption). The agent must verify wound severity, presentation mode, and post context to determine if violation conditions hold or exemptions apply. Same wound, different context, different conclusion.

Many content moderation judgments share this characteristic: rules cannot enumerate all cases, evidence may be incomplete. Moderators must interpret rules in context, verify applicability conditions, and judge whether evidence supports conclusions. Differing rule understanding and judgment scales cause disagreements.

The paper defines rule-governed contextual decision tasks : combining case-specific context and evidence to make justified decisions under given rules.

Challenges: Uneven Reference Support and Process Scoring Gaps

Reference answers have varying rule and evidence support. Training set: 3,395 cases, each annotated by 4 moderators, using quality-control moderator's label and rationale as reference. 20.9% of cases have human disagreement. Disagreements stem from rule interpretation, judgment scale, or evidence interpretation differences, not necessarily labeling errors.

Human agreement does not guarantee sufficient rule and evidence support. Among 2,686 agreed cases, 718 were judged by an offline support evaluation model as having ambiguity or insufficient support. This evaluation provides a reference support signal, not an independent label correctness judgment. Training must combine both signals to adjust reliance on outcome consistency.

Omissions in rule application may be missed by process scoring. Written rules may not expand evidence requirements, boundaries, and exceptions for each scenario, so agents lack concrete verification steps. For the wound example, after identifying visual clues, the agent must still verify severity and exemption conditions.

If process scoring only checks "whether wound identified" and "whether rule cited", trajectories missing key conditions could still score high. Adding process rewards requires evaluation to reflect rule application quality.

AdaptEvo therefore adjusts training feedback by reference support level while supplementing decision knowledge and refining process scoring criteria.

Method: Alternating Training and Evolution

AdaptEvo alternates two phases:

Training: Fix decision knowledge and scoring criteria, update model parameters.

Evolution: Freeze policy, analyze training trajectories, update decision knowledge and process scoring criteria for next phase.

CA-GRPO: Confidence-Adaptive Reward Allocation

CA-GRPO uses two rewards:

Outcome reward: Measures consistency with reference label.

Process reward: Examines evidence gathering, rule application, and decision derivation.

The "reference confidence" measures the support level of the reference judgment , composed of human agreement and external model evaluation. The external model checks whether rules and available evidence can support the reference label and rationale. It is not the agent's self-confidence nor directly task difficulty.

When support is sufficient, training relies more on outcome feedback; when support is weaker, process feedback carries more weight , preserving learning signals from valuable evidence-gathering and rule-application behaviors. Implementation: outcome reward max weight 90%, process feedback always retained. Support evaluation does not rewrite reference labels; process evaluation defaults to not reading original reference answers.

Knowledge and Scoring Criteria Evolution: Refining Judgment Basis from Common Issues

After each training round, AdaptEvo freezes the policy, analyzes cross-case recurring judgment problems using rules and evidence, and synchronously updates decision knowledge and process scoring criteria:

Decision knowledge specifies "what to verify" : distills common issues into guidance containing trigger conditions, necessary evidence, decision boundaries, and exceptions, placed in agent context.

Scoring criteria check "whether verification is thorough" : focus on whether necessary conditions are confirmed, tool evidence is used for decisions, and whether "not found" is mistakenly treated as negative evidence.

Candidate knowledge is validated by re-running the agent; candidate scoring criteria are validated on fixed trajectories for discrimination ability and scoring stability. Verified updates are used in next training phase. Diagnosis and validation use only training data; test sets are not involved; business rules remain unchanged.

Results: Surpassing Gemini 3.8-Flash, Maintaining Generalization After Rule Changes

Evaluated on industrial multimodal content moderation. In-Period test set from same collection period as training data; Out-of-Period test set from later period with updated applicable rules. Metrics: Exact Label Accuracy (ELA) requires predicted label exactly match reference label; Binary Decision Accuracy (BDA) merges violation categories to measure "pass/violate" consistency.

In-Period both metrics surpass Gemini 3.8-Flash, achieving best performance. Based on Qwen3.6-35B-A3B, AdaptEvo reaches 61.9% ELA and 72.2% BDA , exceeding Gemini 3.8-Flash, ranking first among all evaluated models and configurations. Compared to same-backbone GRPO, improvements of 7.5 and 3.7 percentage points respectively.

Training gains persist after rule changes. On Out-of-Period test set with updated rules, AdaptEvo achieves 59.5% ELA and 69.5% BDA , 7.6 and 4.3 percentage points higher than GRPO. ELA is highest among same-base-model training methods; BDA matches Gemini 3.8-Flash, demonstrating generalization to rule changes.

Decision knowledge distilled from training cases directly improves new case judgments. Without updating model parameters, merely injecting evolved knowledge into CA-GRPO at inference improves In-Period metrics by 3.7 and 3.2 percentage points ; continuing training with this knowledge and updated scoring criteria adds further 2.8 and 0.9 percentage points . Summarized conditions, evidence requirements, and exceptions not only guide inference directly but also combine with subsequent training to further improve decision accuracy.

Generalization Analysis and Training Mechanism

When test rules change, can models retain training gains? Experiments show GRPO achieves higher average outcome reward during training, yet Out-of-Period accuracy declines as training progresses : from step 10 to step 60, ELA and BDA drop 2.6 and 0.8 percentage points, both falling below base model. Even training only on high-confidence cases fails to avoid this degradation. This indicates that merely increasing matching to training reference answers, or selecting more reliable reference answers, is insufficient to guarantee generalization after rule changes; training only on high-confidence cases shows similar trend.

In contrast, CA-GRPO maintains Out-of-Period ELA above base model at all evaluated checkpoints , with step 60 ELA/BDA 1.9/1.4 percentage points higher , demonstrating CA-GRPO's advantage in preserving generalization gains during continued training.

Project homepage: https://github.com/AllSpark-Research/AdaptEvo

Paper link: https://arxiv.org/abs/2610.11354