| Autor: |
Shuken SR; Department of Chemistry, Stanford University, 364 Lomita Dr., Stanford, California94305, United States.; Department of Neurology and Neurological Sciences, Stanford University School of Medicine, 291 Campus Dr., Stanford, California94305, United States.; Wu Tsai Neurosciences Institute, Stanford University, 290 Jane Stanford Way, Stanford, California94305, United States., McNerney MW; Mental Illness Research Education and Clinical Center (MIRECC), Veterans Affairs Palo Alto Health Care System, 3801 Miranda Avenue, Palo Alto, California94304, United States.; Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, 291 Campus Dr., Stanford, California94305, United States. |
| Abstrakt: |
The multiple hypothesis testing problem is inherent in large-scale quantitative "omic" experiments such as mass spectrometry-based proteomics. Yet, tools for comparing the costs and benefits of different p -value correction methods under different experimental conditions are lacking. We performed thousands of simulations of omic experiments under a range of experimental conditions and applied correction using the Benjamini-Hochberg (BH), Bonferroni, and permutation-based false discovery proportion (FDP) estimation methods. The tremendous false discovery rate (FDR) benefit of correction was confirmed in a range of different contexts. No correction method can guarantee a low FDP in a single experiment, but the probability of a high FDP is small when a high number and proportion of corrected p -values are significant. On average, correction decreased sensitivity, but the sensitivity costs of BH and permutation were generally modest compared to the FDR benefits. In a given experiment, observed sensitivity was always maintained or decreased by BH and Bonferroni, whereas it was often increased by permutation. Overall, permutation had better FDR and sensitivity than BH. We show how increasing sample size, decreasing variability, or increasing effect size can enable the detection of all true changes while still correcting p -values, and we present basic guidelines for omic experimental design. Analysis of an experimental proteomic data set with defined changes corroborated these trends. We developed an R Shiny web application for further exploration and visualization of these models, which we call the Simulator of P -value Multiple Hypothesis Correction (SIMPLYCORRECT) and a high-performance R package, permFDP, for easy use of the permutation-based FDP estimation method. |
