How MedGuard Embeds Fact‑Checking into Telemedicine to Guard Diagnostic Safety

Problem Statement

In online consultations a simple reassurance such as “your fatty liver is not a concern” can hide factual errors. Traditional safety solutions rely on keyword rules or coarse text classification, which fail when the risk depends on long‑range context, clinical guidelines, and evidence chains.

MedGuard Overview

MedGuard, a joint effort by Ant Group’s AI Safety Lab and Xiamen University, is an LLM‑based gatekeeper that inserts medical fact‑checking into the diagnostic workflow. It is not a chat‑bot but a safety layer that validates key medical claims before a doctor’s opinion, AI‑generated reply, or prescription is delivered.

The system decomposes a multi‑turn doctor‑patient dialogue into atomic medical claims —single statements that can be independently judged true or false (e.g., “the imaging excludes fatty liver”). It extracts patient‑level context (age, sex, history, allergies, current medication) and rewrites dispersed statements to preserve negations, dosages, and temporal constraints.

The verification pipeline consists of four stages: (1) medical claim extraction, (2) patient context construction, (3) claim de‑contextualisation, and (4) quality review with deduplication. Claims are then routed based on uncertainty: clear “No Risk” or “High Risk” claims follow a fast path, while “Low Risk” (insufficient evidence) triggers an evidence‑driven retrieval loop.

Evidence‑Driven Retrieval

When a claim enters the retrieval path, a planner converts it into a structured search task, identifying core entities, indications/contra‑indications, and relevant disease or drug domains. The retriever queries a knowledge base built from six authoritative Chinese medical resources containing 16,535 disease records and 187,738 drug records.

The summariser extracts evidence fragments that retain dosage, contraindication, interaction, and quantitative details, then assesses whether the evidence is sufficient. If not, the planner revises the search strategy and repeats until a satisfactory evidence chain is obtained.

Evaluation

MedGuard was benchmarked on the MedGuardEval suite covering real online consultations, complex clinical cases, and drug‑knowledge scenarios. Using a 7‑billion‑parameter model, it achieved state‑of‑the‑art performance, improving average F1 by 22.1 % over baselines and surpassing open‑source models and HealthCareAgent.

In fine‑grained dialogue‑level risk detection, it also outperformed baselines by 5.8 % on coarse‑grained evaluation.

Clinical assessment by 126 certified professionals across 10 specialties (604 dialogue segments) gave average scores above 4.2 / 5 on factual accuracy, error reduction, legal‑ethical safety, and overall usability.

Retrospective validation on 10,000 real consultations showed risk‑identification variance within ±0.03 across physiological systems, generating 2,234 alerts versus 4,785 from a competing model, thus reducing unnecessary warnings while maintaining safety.

Risk Boundary Judgment

Beyond flagging errors, MedGuard distinguishes between genuine risks and acceptable clinical decisions. In the earlier example of a lymphoma patient, MedGuard identified the false claim that “imaging excludes fatty liver” while preserving the doctor’s valid suggestions for MRI and AFP testing, demonstrating nuanced boundary awareness.

Integration into Clinical Workflow

MedGuard is designed as an embeddable safety infrastructure, not a standalone diagnosis system. It can be invoked at three points:

Pre‑response/Prescription Check: Verify medical facts before a clinician or AI finalizes a recommendation.

Real‑time Consultation Alerts: Promptly warn of potential factual risks during the dialogue.

Historical Quality Review: Retrospectively audit completed consultations for fact‑checking compliance.

The output includes the original claim, risk level, supporting evidence, and explanatory suggestions, providing a “peer reviewer” that backs clinicians with authoritative evidence without disrupting workflow.

Conclusion

MedGuard demonstrates that embedding LLM‑driven, uncertainty‑guided fact‑checking into telemedicine can substantially improve diagnostic safety, reduce false reassurance, and maintain a low false‑alert rate, positioning it as a critical “gatekeeper” for the era of AI‑augmented clinical care.