Do We Really Know What Biomedical Foundation Models Have Learned? Benchmarking Challenges Revealed

Traditional machine learning evaluation relies on held-out test sets from the same distribution, but biomedical foundation models — such as single-cell foundation models (scFMs) — are expected to generalize across tasks, species, platforms, and biological contexts. The article reviews a Nature Methods publication (Heisenberg University, NYU, et al., 2026-09-04, DOI: https://www.nature.com/articles/s41592-026-03182-y) that identifies fundamental gaps in current benchmarking practices.

Limitations of Static Leaderboards

Established challenges like CASP (protein structure) and DREAM (gene regulation) provide valuable task-specific benchmarks, yet they test only a narrow slice of a foundation model's potential. A model may excel on a few selected tasks by overfitting to dataset idiosyncrasies without acquiring transferable biological knowledge. The paper illustrates this with scFMs, which are trained on massive single-cell atlases to learn universal representations of cell states and gene interactions, targeting tasks including cross-species comparison, data integration, missing modality inference, and gene or drug perturbation response prediction.

Why Perturbation Prediction Matters

Single-cell data suffer from systematic technical biases (batch effects, platform differences). The authors argue that evaluating models on unseen genetic or drug perturbations probes whether the model has learned causal regulatory mechanisms rather than static correlations. If a model can predict cellular responses to a perturbation it never encountered during training, it demonstrates mechanistic generalization.

Fine-Tuning vs. Probe Models: A Concrete Comparison

In one case study, a frozen pre-trained scGPT coupled with a random forest probe outperformed full fine-tuning of scGPT on perturbation prediction. This contradicts the assumption that larger models always benefit more from end-to-end fine-tuning, highlighting that evaluation must account for downstream adaptation strategy.

Beyond a Single Score: Recommended Benchmark Components

Simple baselines and null models to quantify trivial performance.

Upper-bound references reflecting irreducible technical and biological variation.

Ablation studies to identify which data modalities, architectural components, or pre-training objectives drive performance.

Agentic Systems and Partial Credit

As biomedical AI evolves into agentic workflows (LLMs planning multi-step tool use), binary success/failure metrics become inadequate. The paper suggests scoring intermediate reasoning steps, assessing reliability, collaboration, and alignment — e.g., detecting whether a model circumvents evaluation rules.

BEACON: A Community-Driven Benchmark Ecosystem

The authors introduce BEACON (Benchmarking, Evaluation and Assessment CONsortium), a cross-disciplinary initiative to unify fragmented benchmarking efforts, standardize protocols, and enable continuous, reproducible evaluation. The goal is to replace monolithic leaderboards with a living benchmark infrastructure that maps the true generalization boundaries of scientific foundation models.