Insights from an autism imaging biomarker challenge: Promises and threats to biomarker discovery

Autor: Monique Elmaleh, Freddy Cliquet, Nicolas Guigui, David Germanaud, Ayoub Ghriss, Balázs Kégl, Anita Beggiato, Roberto Toro, Richard Delorme, Amicie de Pierrefeu, Valentina Zantedeschi, Joris Van den Bossche, Nicolas Traut, Laurent Bonasse-Gahot, Alexandre Boucaud, Alban Bethegies, Guillaume Lemaitre, Thomas Bourgeron, Katja Heuer, Meng Wang, Gael Varauquaux, Weidong Cai, Stanislas Chambon
Přispěvatelé: Institut Pasteur [Paris], Centre de Recherche Interdisciplinaire / Center for Research and Interdisciplinarity [Paris, France] (CRI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Paris Cité (UPC), Max Planck Institute for Human Cognitive and Brain Sciences [Leipzig] (IMPNSC), Max-Planck-Gesellschaft, Méthodes computationnelles et mathématiques pour comprendre la société et la santé à partir de données (SODA), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), Université Paris-Saclay, AP-HP Hôpital universitaire Robert-Debré [Paris], Assistance publique - Hôpitaux de Paris (AP-HP) (AP-HP), Service NEUROSPIN (NEUROSPIN), Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Université Paris-Saclay, hosa.io, Centre d'Analyse et de Mathématique sociales (CAMS), École des hautes études en sciences sociales (EHESS)-Centre National de la Recherche Scientifique (CNRS), Stanford School of Medicine [Stanford], Stanford Medicine, Stanford University-Stanford University, rythm.co, University of Colorado [Boulder], University of Chinese Academy of Sciences [Beijing] (UCAS), Institute of Automation - Chinese Academy of Sciences, Laboratoire Hubert Curien [Saint Etienne] (LHC), Institut d'Optique Graduate School (IOGS)-Université Jean Monnet [Saint-Étienne] (UJM)-Centre National de la Recherche Scientifique (CNRS), HUAWEI Technologies France (HUAWEI), Montreal Neurological Institute and Hospital, McGill University = Université McGill [Montréal, Canada], Département de Neuroscience - Department of Neuroscience, Institut Pasteur [Paris] (IP)-Centre National de la Recherche Scientifique (CNRS)-Université Paris Cité (UPCité), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Paris Cité (UPCité), Université Paris-Saclay-Direction de Recherche Fondamentale (CEA) (DRF (CEA)), Commissariat à l'énergie atomique et aux énergies alternatives (CEA)-Commissariat à l'énergie atomique et aux énergies alternatives (CEA), Université Paris sciences et lettres (PSL), Institut d'Optique Graduate School (IOGS)-Université Jean Monnet - Saint-Étienne (UJM)-Centre National de la Recherche Scientifique (CNRS), Huawei Technologies France, Huawei Technologies France [Boulogne-Billancourt], Lassailly-Bondaz, Anne, Laboratoire Hubert Curien (LHC)
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Imaging biomarker
Computer science
[SDV.IB.IMA]Life Sciences [q-bio]/Bioengineering/Imaging
Autism Spectrum Disorder
Autism
[SDV.MHEP.PSM] Life Sciences [q-bio]/Human health and pathology/Psychiatrics and mental health
[INFO.INFO-IM] Computer Science [cs]/Medical Imaging
diagnostic
Overfitting
computer.software_genre
MESH: Magnetic Resonance Imaging
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]
benchmark
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML]
Biomarker discovery
MESH: Autism Spectrum Disorder
[SDV.NEU.PC]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]/Psychology and behavior
Brain
Replicate
Magnetic Resonance Imaging
machine learning
Neurology
Autism spectrum disorder
[SDV.NEU]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]
Cognitive Neuroscience
MESH: Autistic Disorder
Machine learning
MESH: Brain
overfit
[INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]
medicine
[INFO.INFO-IM]Computer Science [cs]/Medical Imaging
Humans
[SDV.NEU] Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]
Autistic Disorder
Modalities
MESH: Humans
business.industry
prediction
[INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG]
medicine.disease
[SDV.IB.IMA] Life Sciences [q-bio]/Bioengineering/Imaging
Sample size determination
[SDV.MHEP.PSM]Life Sciences [q-bio]/Human health and pathology/Psychiatrics and mental health
MESH: Biomarkers
Artificial intelligence
[INFO.INFO-BI]Computer Science [cs]/Bioinformatics [q-bio.QM]
business
computer
Biomarkers
Zdroj: NeuroImage
NeuroImage, Elsevier, 2022, 255, pp.119171. ⟨10.1016/j.neuroimage.2022.119171⟩
NeuroImage, 2022, 255, pp.119171. ⟨10.1016/j.neuroimage.2022.119171⟩
ISSN: 1053-8119
1095-9572
Popis: MRI has been extensively used to identify anatomical and functional differences in Autism Spectrum Disorder (ASD). Yet, many of these findings have proven difficult to replicate because studies rely on small cohorts and are built on many complex, undisclosed, analytic choices. We conducted an international challenge to predict ASD diagnosis from MRI data, where we provided preprocessed anatomical and functional MRI data from > 2,000 individuals. Evaluation of the predictions was rigorously blinded. 146 challengers submitted prediction algorithms, which were evaluated at the end of the challenge using unseen data and an additional acquisition site. On the best algorithms, we studied the importance of MRI modalities, brain regions, and sample size. We found evidence that MRI could predict ASD diagnosis: the 10 best algorithms reliably predicted diagnosis with AUC∼0.80 – far superior to what can be currently obtained using genotyping data in cohorts 20-times larger. We observed that functional MRI was more important for prediction than anatomical MRI, and that increasing sample size steadily increased prediction accuracy, providing an efficient strategy to improve biomarkers. We also observed that despite a strong incentive to generalise to unseen data, model development on a given dataset faces the risk of overfitting: performing well in cross-validation on the data at hand, but not generalising. Finally, we were able to predict ASD diagnosis on an external sample added after the end of the challenge (EU-AIMS), although with a lower prediction accuracy (AUC=0.72). This indicates that despite being based on a large multisite cohort, our challenge still produced biomarkers fragile in the face of dataset shifts.
Databáze: OpenAIRE
Externí odkaz: