To Tweak or Not to Tweak. How Exploiting Flexibilities in Gene Set Analysis Leads to Overoptimism.

Autor: Wünsch M; Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine, LMU Munich, Munich, Germany.; Munich Center for Machine Learning, Munich, Germany., Sauer C; Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine, LMU Munich, Munich, Germany.; Munich Center for Machine Learning, Munich, Germany., Herrmann M; Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine, LMU Munich, Munich, Germany.; Munich Center for Machine Learning, Munich, Germany., Hinske LC; Institute for Digital Medicine, University Hospital of Augsburg, Augsburg, Germany., Boulesteix AL; Institute for Medical Information Processing, Biometry, and Epidemiology, Faculty of Medicine, LMU Munich, Munich, Germany.; Munich Center for Machine Learning, Munich, Germany.
Jazyk: angličtina
Zdroj: Biometrical journal. Biometrische Zeitschrift [Biom J] 2025 Feb; Vol. 67 (1), pp. e70016.
DOI: 10.1002/bimj.70016
Abstrakt: Gene set analysis, a popular approach for analyzing high-throughput gene expression data, aims to identify sets of genes that show enriched expression patterns between two conditions. In addition to the multitude of methods available for this task, users are typically left with many options when creating the required input and specifying the internal parameters of the chosen method. This flexibility can lead to uncertainty about the "right" choice, further reinforced by a lack of evidence-based guidance. Especially when their statistical experience is scarce, this uncertainty might entice users to produce preferable results using a "trial-and-error" approach. While it may seem unproblematic at first glance, this practice can be viewed as a form of "cherry-picking" and cause an optimistic bias, rendering the results nonreplicable on independent data. After this problem has attracted a lot of attention in the context of classical hypothesis testing, we now aim to raise awareness of such overoptimism in the different and more complex context of gene set analyses. We mimic a hypothetical researcher who systematically selects the analysis variants yielding their preferred results, thereby considering three distinct goals they might pursue. Using a selection of popular gene set analysis methods, we tweak the results in this way for two frequently used benchmark gene expression data sets. Our study indicates that the potential for overoptimism is particularly high for a group of methods frequently used despite being commonly criticized. We conclude by providing practical recommendations to counter overoptimism in research findings in gene set analysis and beyond.
(© 2024 The Author(s). Biometrical Journal published by Wiley‐VCH GmbH.)
Databáze: MEDLINE
Externí odkaz: