dnaHNet Boosts Inference Speed 3× and Cuts Genomic Learning Cost by Nearly 4×

Genomic sequences encode the complete genetic information of organisms, and deciphering the hidden "DNA grammar" is a central challenge for modern biology, impacting disease diagnosis, drug discovery, and synthetic biology.

Recent large‑scale pre‑trained models such as Nucleotide Transformer and Evo have demonstrated performance gains following a scaling law, but DNA’s continuous, boundary‑less nature creates a trade‑off between fixed‑token or single‑nucleotide modeling: fixed tokenization can break functional units, while nucleotide‑level modeling incurs high computational cost.

To address this, researchers from the University of Toronto, Canada’s Vector AI Institute, and the U.S. Arc Institute proposed dnaHNet, a scalable hierarchical foundation model for genomic sequence learning (arXiv:2602.10603). The model introduces a dynamic chunking mechanism that lets the network learn context‑dependent biological tokens, avoiding explicit tokenizers.

Multi‑level Genomic Dataset Design

The training data are derived from a curated subset of the GTDB genome taxonomy database, following the filtering, quality‑control, and deduplication pipeline of the Evo OpenGenome dataset. After filtering for assembly completeness, contamination, and marker gene content, one representative genome per species‑level cluster is retained, yielding 85,205 prokaryotic species, 17,648,721 sequences, and roughly 1.44 × 10¹² nucleotides, split into non‑overlapping 8,192‑nt fragments.

Evaluation uses three complementary test sets: (1) local coding fitness assessed with 12 E. coli K‑12 MaveDB experiments (21,250 data points); (2) whole‑genome functional judgment via binary essentiality labels for 62 bacteria from the DEG database (185,226 points); and (3) structural interpretability examined on the Bacillus subtilis genome, aligning model‑derived segments with annotated functional regions.

dnaHNet Architecture

dnaHNet treats genomic learning as an autoregressive nucleotide prediction task. It employs a hierarchical structure where each layer contains an encoder, a backbone, and a decoder. The encoder uses a routing mechanism to detect positions of significant information change (e.g., codon boundaries) and compresses the sequence into implicit chunks. The backbone combines Mamba and Transformer modules to capture long‑range dependencies while preserving key local information. The decoder upsamples the representation back to nucleotide resolution for prediction.

Approximately 30 % of model capacity is allocated to the encoder and decoder to enhance local structure modeling. A two‑stage hierarchical compression is applied: the first stage captures short‑scale patterns such as codons, and the second stage models longer functional structures, balancing compression efficiency with information fidelity. Training jointly optimizes autoregressive loss and a compression‑rate constraint.

During inference, the model dynamically determines chunk boundaries based on learned boundary probabilities, allowing adaptive granularity that mirrors true genomic organization.

Computational Cost and Scaling

Scaling experiments trained over 100 models of varying sizes under a fixed compute budget (total FLOPs ≈ 8 × 10¹⁹). With a sequence length of 10⁶ nucleotides, the 218 M‑parameter dnaHNet reduces computational cost by about 3.89 × compared to the 166 M‑parameter StripedHyena2, and the two‑stage version outperforms a single‑stage baseline.

Power‑law fits of perplexity versus compute show that StripedHyena2 requires roughly 3.75 × more FLOPs to reach dnaHNet’s performance level. Moreover, dnaHNet can train on up to 140 B tokens under the same compute budget, whereas competing models converge around 68 B tokens.

Downstream Performance

On zero‑shot variant effect prediction (MaveDB) and gene essentiality classification (DEG), dnaHNet consistently surpasses StripedHyena2 and Transformer++ models, with the gap widening as compute increases. Structural interpretability analysis on the B. subtilis genome reveals that the first compression stage is sensitive to codon patterns, while the second stage emphasizes functional regions such as promoters, start codons, and inter‑genic spaces.

These results demonstrate that dnaHNet not only achieves high‑performance prediction but also reconstructs functional genomic organization in an unsupervised manner, offering an interpretable computational pathway toward decoding DNA grammar.

Conclusion

By eliminating predefined sequence segmentation and allowing the model to learn dynamic chunks, dnaHNet improves both efficiency and fidelity to the multi‑scale structure of genomes. If the model can reliably capture biologically meaningful units, it may unlock new insights for variant prediction, functional discovery, and synthetic design.