NCBench: providing an open, reproducible, transparent, adaptable, and continuous benchmark approach for DNA-sequencing-based variant calling.

Autor:	Hanssen F; Quantitative Biology Center, Eberhard Karls University Tübingen, Tübingen, Germany., Gabernet G; Quantitative Biology Center, Eberhard Karls University Tübingen, Tübingen, Germany., Bäuerle F; Quantitative Biology Center, Eberhard Karls University Tübingen, Tübingen, Germany.; M3 Research Center, University Hospital, Tübingen, Germany.; Institute for Translational Bioinformatics, University Medical Center, Tübingen, Germany.; Institute for Bioinformatics and Medical Informatics (IBMI), Eberhard-Karls University of Tübingen, Tübingen, Germany., Stöcker B; Bioinformatics and Computational Oncology, Institute for Artificial Intelligence in Medicine (IKIM), University Medicine Essen, University of Duisburg-Essen, Essen, Germany., Wiegand F; Bioinformatics and Computational Oncology, Institute for Artificial Intelligence in Medicine (IKIM), University Medicine Essen, University of Duisburg-Essen, Essen, Germany., Smith NH; TUM School of Computation, Information and Technology, Technical University of Munich, Munich, Germany., Mertes C; TUM School of Computation, Information and Technology, Technical University of Munich, Munich, Germany.; Munich Data Science Institute, Technical University of Munich, Munich, Germany.; Institute of Human Genetics, Klinikum rechts der Isar, School of Medicine, Technical University of Munich, Munich, Germany., Neogi AG; Cologne Center for Genomics, University of Cologne, Cologne, Germany., Brandhoff L; Cologne Center for Genomics, University of Cologne, Cologne, Germany.; West German Genome Center - Cologne, University of Cologne, Cologne, Germany., Ossowski A; Cologne Center for Genomics, University of Cologne, Cologne, Germany., Altmueller J; Cologne Center for Genomics, University of Cologne, Cologne, Germany.; Core Facility Genomics, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany.; Max Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Berlin, Germany., Becker K; Cologne Center for Genomics, University of Cologne, Cologne, Germany., Petzold A; DRESDEN-concept Genome Center, TUD Dresden University of Technology, Dresden, Germany., Sturm M; Institute of Medical Genetics and Applied Genomics, University Hospital Tuebingen, Tübingen, Germany., Stöcker T; Institute of Crop Science and Resource Conservation, University of Bonn, Bonn, Germany., Sivalingam S; Institute of Human Genetics, Medical Faculty and University Hospital Düsseldorf, Heinrich-Heine-University Düsseldorf, Düsseldorf, Germany., Brand F; Institute for Genomic Statistics and Bioinformatics, Medical Faculty, University of Bonn, Bonn, Germany., Schmidt A; Institute of Human Genetics, University Hospital of Bonn, Bonn, Germany., Buness A; Core Unit for Bioinformatics Analysis, University Hospital Bonn, Bonn, Germany., Probst AJ; Environmental Metagenomics, Research Center One Health Ruhr, University Alliance Ruhr, Faculty of Chemistry, University of Duisburg-Essen, Essen, Germany., Motameny S; Cologne Center for Genomics, University of Cologne, Cologne, Germany.; West German Genome Center - Cologne, University of Cologne, Cologne, Germany., Köster J; Bioinformatics and Computational Oncology, Institute for Artificial Intelligence in Medicine (IKIM), University Medicine Essen, University of Duisburg-Essen, Essen, Germany.; German Cancer Consortium, Essen, Germany.
Jazyk:	angličtina
Zdroj:	F1000Research [F1000Res] 2024 Sep 12; Vol. 12, pp. 1125. Date of Electronic Publication: 2024 Sep 12 (Print Publication: 2023).
DOI:	10.12688/f1000research.140344.2
Abstrakt:	We present the results of the human genomic small variant calling benchmarking initiative of the German Research Foundation (DFG) funded Next Generation Sequencing Competence Network (NGS-CN) and the German Human Genome-Phenome Archive (GHGA). In this effort, we developed NCBench, a continuous benchmarking platform for the evaluation of small genomic variant callsets in terms of recall, precision, and false positive/negative error patterns. NCBench is implemented as a continuously re-evaluated open-source repository. We show that it is possible to entirely rely on public free infrastructure (Github, Github Actions, Zenodo) in combination with established open-source tools. NCBench is agnostic of the used dataset and can evaluate an arbitrary number of given callsets, while reporting the results in a visual and interactive way. We used NCBench to evaluate over 40 callsets generated by various variant calling pipelines available in the participating groups that were run on three exome datasets from different enrichment kits and at different coverages. While all pipelines achieve high overall quality, subtle systematic differences between callers and datasets exist and are made apparent by NCBench.These insights are useful to improve existing pipelines and develop new workflows. NCBench is meant to be open for the contribution of any given callset. Most importantly, for authors, it will enable the omission of repeated re-implementation of paper-specific variant calling benchmarks for the publication of new tools or pipelines, while readers will benefit from being able to (continuously) observe the performance of tools and pipelines at the time of reading instead of at the time of writing. Competing Interests: No competing interests were disclosed. (Copyright: © 2024 Hanssen F et al.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu