Nucleotide Transformer: building and evaluating robust foundation models for human genomics.

Autor: Dalla-Torre H; InstaDeep, London, UK., Gonzalez L; InstaDeep, London, UK., Mendoza-Revilla J; InstaDeep, London, UK., Lopez Carranza N; InstaDeep, London, UK., Grzywaczewski AH; Nvidia, Santa Clara, CA, USA., Oteri F; InstaDeep, London, UK., Dallago C; Nvidia, Santa Clara, CA, USA.; Technical University of Munich, Munich, Germany., Trop E; InstaDeep, London, UK., de Almeida BP; InstaDeep, London, UK., Sirelkhatim H; Nvidia, Santa Clara, CA, USA., Richard G; InstaDeep, London, UK., Skwark M; InstaDeep, London, UK., Beguir K; InstaDeep, London, UK., Lopez M; InstaDeep, London, UK. m.lopez@instadeep.com., Pierrot T; InstaDeep, London, UK. t.pierrot@instadeep.com.
Jazyk: angličtina
Zdroj: Nature methods [Nat Methods] 2024 Nov 28. Date of Electronic Publication: 2024 Nov 28.
DOI: 10.1038/s41592-024-02523-z
Abstrakt: The prediction of molecular phenotypes from DNA sequences remains a longstanding challenge in genomics, often driven by limited annotated data and the inability to transfer learnings between tasks. Here, we present an extensive study of foundation models pre-trained on DNA sequences, named Nucleotide Transformer, ranging from 50 million up to 2.5 billion parameters and integrating information from 3,202 human genomes and 850 genomes from diverse species. These transformer models yield context-specific representations of nucleotide sequences, which allow for accurate predictions even in low-data settings. We show that the developed models can be fine-tuned at low cost to solve a variety of genomics applications. Despite no supervision, the models learned to focus attention on key genomic elements and can be used to improve the prioritization of genetic variants. The training and application of foundational models in genomics provides a widely applicable approach for accurate molecular phenotype prediction from DNA sequence.
Competing Interests: Competing interests: H.D.-T., L.G., J.M.-R., N.L.C., F.O., E.T., B.P.d.A., G.R., M.S., K.B., M.L. and T.P. are employees of InstaDeep. A.H.G. and C.D. are employees of Nvidia.
(© 2024. The Author(s).)
Databáze: MEDLINE
Externí odkaz: