Closing the Legal Judgment Gap: Evidence‑Based Fact Prediction (LFP) & LFPBench

Abstract

Legal judgment prediction (LJP) traditionally predicts outcomes from court‑determined facts. In early litigation only evidentiary items are available, creating a logical and temporal mismatch. Legal Fact Prediction (LFP) reframes the task: given a set of evidentiary items, predict the underlying legal facts that will later inform the judgment.

Evolution of Legal Judgment Prediction

Statistical machine learning : SVM‑based win‑loss classifiers.

Deep learning : Hierarchical Attention Networks (HAN) for charge and sentence prediction (e.g., CAIL‑2018).

Pre‑trained language models : BERT, RoBERTa achieving state‑of‑the‑art scores.

LLM and generative AI : Current stage exploring explainable, logically complex generation.

Medvedeva and McBride (2023) criticize the field for reinforcing judicial bias and for the evidence‑vs‑fact input paradox.

Legal Fact Prediction Paradigm

LFP requires reconstructing a complete factual picture from fragmented, possibly contradictory evidence, unlike legal document summarization which compresses text. The task demands multi‑party perspective handling and logical inference from raw evidentiary items.

LFPBench Dataset Construction

LFPBench is built from Chinese civil case documents by extracting evidence‑fact pairs. Each entry contains anonymized evidence, fact annotations, and metadata describing evidence order and quantity. The dataset deliberately limits geographic scope to Chinese civil law to ensure consistent legal reasoning while preserving research utility.

Experimental Evaluation

Models evaluated on LFPBench include GPT‑4o, Claude 3.5 Sonnet, and several domain‑specific legal LLMs. Key results:

Introducing the LFP task reduces the average accuracy loss between evidence‑based and fact‑based prediction by 38.5 % .

General‑purpose LLMs recover roughly 40 % of the performance gap.

All models exhibit systematic weaknesses in (a) handling conflictual evidence, (b) long‑text reasoning, and (c) bias toward the training jurisdiction.

Discussion

Flexible Input Adaptation

The LFP framework can ingest different litigation‑stage inputs (complaint, answer, proof). Future systems should dynamically select the appropriate evidence subset—for example, using only the complaint for early assessment or the full trial transcript for precise prediction.

Ethical Considerations

Two major challenges are identified:

Judicial bias : Training data inherit historical biases from past judgments, which LFP cannot fully eliminate.

Privacy : Even with anonymization, real‑world deployment must protect sensitive personal information, especially in family‑law cases.

Limitations

Geographic scope : All data originate from Chinese civil law; models may not generalize to common‑law jurisdictions that rely on precedent.

Information loss : Models learn from textual summaries of evidence rather than raw artifacts (e.g., scanned contracts, video recordings), creating a gap between training substrate and real‑world evidence.

Conclusion

Empirical analysis confirms that LFP is a necessary processing step linking raw evidence to final judgments. While large general‑purpose models can close about 40 % of the evidence‑to‑fact performance gap, they remain weak in complex logical reasoning, adversarial evidence handling, and bias mitigation. Advancing legal AI therefore requires enhancing long‑range reasoning, multimodal evidence integration, and deeper legal‑logic understanding.

References

arXiv:2409.07055 – Junkai Liu et al., “Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction.”

Lawlor (1963) – early work on legal judgment prediction.

Medvedeva and McBride (2023) – critique of bias and methodological gaps in LJP.

Code example

作者：张瀚元
本文
约3000字
，建议阅读
5
分钟
本文介绍了LFP全新范式，剖析法律AI研究局限并展望未来发展方向。