Can Peer Review Boost Large Language Model Ensembles? Introducing LLM‑PeerReview

Background and Motivation

With the rapid growth of open‑source large language models (LLMs) such as Llama, Qwen, and DeepSeek, Hugging Face now hosts over 180,000 models, yet no single model excels on every task. Some models are strong at mathematical reasoning, others at code generation, and still others at open‑domain QA. This diversity motivates research into LLM ensembles that can vote or collaborate to produce the best answer.

Limitations of Existing Ensemble Methods

Fine‑tuning‑based generation (e.g., LLM‑Blender) : Requires large labeled datasets and additional training, limiting generalization to new tasks.

Similarity‑based selection (e.g., Smoothie, Agent‑Forest) : Relies on coarse similarity metrics like BLEU, which cannot capture subtle quality differences and are vulnerable to hallucinations.

LLM‑PeerReview: An Unsupervised Peer‑Review Inspired Framework

The proposed LLM‑PeerReview framework draws inspiration from academic peer review and operates without any supervision or fine‑tuning. It consists of three sequential modules:

Scoring : Each LLM in a model pool acts as a judge (LLM‑as‑a‑Judge) and assigns a score to every candidate response for a given prompt/query. To mitigate judge bias, the authors introduce the flipped‑triple scoring trick , a key technique that improves scoring reliability.

Reasoning : Scores from multiple judges are aggregated. Two variants are offered: LLM‑PeerReview (simple averaging) and LLM‑PeerReview‑W (weight‑aware averaging based on each judge’s perceived expertise).

Selection : For each prompt/query, the response with the highest aggregated score is selected as the final output.

Key Technical Details

The flipped‑triple scoring trick replaces the traditional point‑wise scoring (where a single model scores each answer independently) with a three‑step process that reduces fixed biases, especially for medium‑sized models. The weighted variant (LLM‑PeerReview‑W) further refines performance by assigning higher influence to more reliable judges.

Advantages of the framework include:

Fully unsupervised – no labeled data or fine‑tuning required.

High interpretability – the scoring process is transparent and can be examined via a transition matrix.

Versatility – applicable to exact‑match generation tasks (e.g., math) and open‑ended tasks (e.g., code generation, instruction following).

Efficiency and Theoretical Insights

Scoring can be parallelized across any subset of judges, allowing linear reduction of computational cost by decreasing the number of judges. Compared with multi‑round debate‑style collaboration methods, LLM‑PeerReview requires only a single scoring round, offering better efficiency.

The authors also provide theoretical arguments showing that increasing the number of judges or their diversity improves the quality of the aggregated score, guiding the selection of judges.

Experimental Evaluation

Experiments on a suite of benchmarks (knowledge QA, mathematical reasoning, instruction following) demonstrate that LLM‑PeerReview and its weighted variant consistently outperform:

Any single LLM in the pool.

All existing LLM‑Ensemble baselines.

State‑of‑the‑art post‑hoc integration methods such as Smoothie‑Global and GaC.

Key findings include:

Average performance gains of 6.9%–7.3% over Smoothie‑Global and 7.2%–7.6% over GaC.

Significant improvements (4–8%) when using the flipped‑triple scoring trick versus traditional point‑wise scoring.

Even with a single judge, the flipped‑triple variant achieves competitive results, and performance scales with more judges.

The weighted version (LLM‑PeerReview‑W) yields modest additional gains, suggesting future work could incorporate richer priors.

Additional Analyses

Further analyses explore the impact of judge count, efficiency trade‑offs, and the relationship between LLM‑Ensemble and broader LLM‑Collaboration research. The authors argue that ensemble methods are a subset of collaboration approaches, but ensembles focus on end‑to‑end query processing rather than extensive inter‑model communication.

Conclusion and Future Directions

LLM‑PeerReview demonstrates that mimicking human peer review—an unsupervised, interpretable, and flexible scoring‑reasoning‑selection pipeline—can substantially improve LLM ensemble performance across diverse tasks. Limitations include the current focus on selecting a single best answer; future work may explore generation‑based ensembles that synthesize multiple answers or incorporate human feedback for hybrid human‑AI review loops.

**Paper Title:** Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer‑Review Process
**Paper Link:** https://arxiv.org/abs/2512.23213
**GitHub:** https://github.com/zeyuji/LLM-PeerReview
**Project Page:** https://zeyuji.github.io/LLM-PeerReview/