How AI Learned to Read the Genomic “Dialects” of 300,000 People for Precise Expression Prediction

From a "Standard Textbook" to Real‑World Genomic Language

In functional genomics, accurately predicting gene expression is hampered by models that are trained solely on a single reference human genome, analogous to learning a language from only one textbook and missing regional accents and dialects.

Such models, including the original Enformer, cannot capture the millions of individual‑specific variants that drive expression differences and disease susceptibility.

Two‑Stage Framework that Merges Global and Local Information

The authors first pre‑trained a BERT‑style model, UKBioBERT , on the UK Biobank dataset, which contains ~300 k European‑ancestry individuals and more than 13 million single‑nucleotide polymorphisms (SNPs) and other variants. During training, the sequence data were not limited to the reference genome; the algorithm simulated realistic “substitutions”, “insertions”, and “deletions” to expose the model to genuine population‑level variation.

UKBioBERT therefore learns a variant‑aware DNA embedding that reflects the rich “context” of each allele in a population.

In the second stage, the UKBioBERT embeddings are fused with the state‑of‑the‑art sequence‑to‑function model Enformer to construct UKBioFormer . The fusion does not replace Enformer; it augments it by injecting the population‑aware embeddings while preserving Enformer’s ability to capture long‑range genomic interactions. Parameter‑efficient fine‑tuning is applied to keep computational costs low.

Better Representations, More Accurate Predictions, Deeper Insights

Evaluation shows that gene embeddings from UKBioBERT separate functional gene categories more cleanly than those from the baseline model, demonstrating superior capture of biological function.

When the embeddings are supplied to the cell‑line expression predictor EPInformer, prediction accuracy improves consistently across multiple cell lines (e.g., K562, GM12878), confirming the general utility of the learned representations.

On the GTEx individualized expression prediction task, UKBioFormer outperforms the original Enformer, a personalized fine‑tuned Performer, and the traditional ElasticNet baseline. The advantage is especially pronounced for genes that already exhibit moderate predictability.

Cross‑population tests reveal that a model trained on European‑ancestry data retains robust performance on African‑ancestry test sets, indicating reduced bias from population‑specific training data.

Summary and Outlook

By injecting large‑scale population variant information into the pre‑training of a genomic language model, the study builds a bridge between “sequence‑context learning” and “individualized functional prediction”. The UKBioBERT and UKBioFormer models demonstrate that moving beyond a single reference genome toward a “genomic encyclopedia” of diverse human variation is essential for more accurate and personalized functional genomics.

Future work will expand the diversity of training cohorts, further diminish population bias, and apply these models to disease cohorts to elucidate how genetic variation drives complex disease mechanisms, ultimately supporting precision medicine.

Reference : Liu, T., Zhang, X., Lin, J. et al. Pre‑training genomic language model with variants for better modeling functional genomics. npj Artificial Intelligence 2, 46 (2026).