Resolving Multi-Agent Disagreements: Align Context, Categorize Conflict, Set Verification Boundaries

The article uses a procurement review scenario where one agent approves a supplier while another flags an expired certificate to illustrate a systematic approach for handling multi-agent disagreements.

First, Align on What Is Being Compared

Before attempting resolution, verify that both agents are evaluating the same thing. Check five alignment points:

Task object — are they reviewing the same legal entity (group vs. subsidiary)?

Material version — is one reading an old attachment while the other sees a newly submitted certificate?

Query time — does the check reflect the current valid state?

Rule version — which regulation version applies?

Authorization scope — what permissions does each agent have?

If these conditions differ, reconcile them first. If original materials are lost and only agent summaries remain, re-fetch primary evidence; do not treat mutually referenced summaries as independent sources.

Classify the Disagreement Type and Apply the Matching Resolution

The article provides a five-category framework with concrete procurement examples:

Object or version mismatch — Example: comparing old vs. new qualification certificates. Resolution: re-align object, time, and version, then re-check.

Factual inconsistency — Example: documents say valid; authoritative system shows expired. Resolution: verify source, update timestamp, and original evidence.

Rule inconsistency — Example: general rule requires supplementary material; project claims exception. Resolution: check valid approvals and rule applicability scope.

Trade-off difference — Example: submit missing material first vs. process other items first. Resolution: decision by person with responsibility and authority.

Insufficient evidence — Example: no confirmation of certificate validity found. Resolution: preserve unknown, stop actions dependent on that fact.

Automatable checks (expiry dates, required fields, approval numbers) can be delegated to code; policy exceptions require documented approval evidence. Separating these two classes eliminates many rounds of model debate.

Majority Vote Does Not Fix Shared Misunderstanding

Three agents reading the same outdated document may unanimously reach the wrong conclusion; swapping models does not guarantee correction. Multi-agent analysis adds diverse checking angles, but collapsing results to a vote buries source quality and authority issues. Two reports derived from the same source are not two independent pieces of evidence. The article cites Anthropic's multi-agent research system discussion on evaluating results and processes, emphasizing the need to verify: original certificate found, cited rules applicable, and re-verification actually resolved the dispute. In procurement, three "recommend approve" votes cannot substitute for the missing qualification itself.

Set Explicit Boundaries for Re-verification

After detecting a conflict, the system may re-fetch original attachments and query business systems once. If inconsistency persists, escalate rather than loop indefinitely. Define in advance: maximum re-verification attempts, time budget, and allowed data sources. Simple issues need no complex workflow; but when a qualification remains unresolved, exhausting the budget means pause — not pick an answer and proceed. Non-dependent, uncontested work (e.g., cataloging documents) may continue, but submission of the purchase request must wait for the qualification decision. Progress splitting depends on task dependencies, not agent willingness.

Record Decisions, Not Just Chat Logs

Handing over a raw conversation thread is insufficient. Structure a disagreement-handling record covering:

Conflict & Impact — Which judgments clash, and what action is blocked?

Bases Cited — What materials, rules, and versions did each side reference?

Verification Performed — Which sources were checked, what were the results, what remains unknown?

Final Disposition — Proceed, request supplement, pause, or apply approved exception?

Decision Authority — Who decided, under what authority, with what scope and validity period?

A human choice for a specific project risk does not automatically become a general rule for future projects. If the root cause was an agent reading an old attachment, fix the material ingestion; if a genuine rule conflict exists, clarify approval responsibility. The goal is resolving the underlying problem, not merely making agents output the same sentence.

Reference

Anthropic, How we built our multi-agent research system : https://www.anthropic.com/engineering/multi-agent-research-system