How Large Discovery Models Enable AI to Design the Next Experiment

Large Discovery Models (LDMs)

Architecture : LDMs combine a generative foundation model p_\theta(x) with a Gaussian‑process (GP) reward model. The generator expands the candidate space using historical context, while the GP builds a Bayesian reward signal from experimental data to evaluate each candidate for the next iteration.

Fast and Slow Learning Loops

Fast loop (per‑experiment): the LLM generates open‑ended candidates, the GP predicts expected reward and uncertainty, and an acquisition function (e.g., Upper‑Confidence Bound (UCB), Expected Improvement (EI), or multi‑objective Expected Hypervolume Improvement (EHVI)) selects the most promising batch for costly evaluation.

Slow loop : high‑value decisions from the fast loop are distilled back into the model parameters via supervised fine‑tuning (SFT), enabling the model to learn when to exploit, explore, or discover new regions.

Core Formulation

p_{\theta,\alpha}(x) \propto p_{\theta}(x) \exp\bigl(\eta\,a_t(x)\bigr)

where p_{\theta} is the generative prior, a_t(x) is the GP‑derived acquisition value, \alpha are inference parameters, and \eta controls the strength of the reward signal. This distribution preserves diversity while steering search toward high‑value regions.

Experimental Validation (v0.1)

Auto‑Research (neural‑network training) : LDM reduced the BPB metric by a factor of 2.4× compared with a pure LLM reflective baseline, achieving a BPB of 0.902291 on B200 hardware and ranking first on the Auto‑Research leaderboard.

Antibody design : Across five antigens, each with a budget of 200 evaluations, LDM lowered average binding‑energy scores by 18.2% relative to pure LLM approaches and discovered novel CDRH3 motifs within a 20^{11} ‑size sequence space.

Small‑molecule design : Optimizing docking and neural‑network activity for KRAS G12D, LDM increased Pareto hyper‑volume by >60% over pure LLM and >63% over traditional multi‑objective Bayesian optimization using only 80 evaluations.

Cross‑Task Generalization

Two‑level out‑of‑distribution tests removed all antibody data during training. The fine‑tuned model matched or exceeded a model that had seen antibody data on four of five antigens, indicating that the learned “design‑from‑experience” capability transfers across domains.

Failure Analysis

The only clear failure occurred on antigen 1H0D_C, where the optimal region was extremely narrow and required specific aromatic/hydrophobic motifs that the model could not infer without domain‑specific knowledge.

Limitations and Future Work

Current evaluations rely on simulated or in‑silico assays rather than wet‑lab experiments. GP scalability can be challenged in very high‑dimensional, non‑stationary settings. The authors plan to address these issues and expand LDM training to additional scientific domains.

References

Paper: https://arxiv.org/abs/2608.15669

Project website: https://largediscovery.net/

GitHub repository: https://github.com/yzailab/Large-Discovery-Models

Hugging Face collection: https://huggingface.co/collections/Yangtze-ailab/large-discovery-model-v01