| Popis: |
Background: Evaluating the performance of metagenomics analyses has proven a challenge, due in part to limited ground-truth standards, broad application space, and numerous evaluation methods and metrics. Application of traditional clinical performance metrics (i.e. sensitivity, specificity, etc.) using taxonomic classifiers do not fit the “one-bug-one-test” paradigm. Ultimately, users need methods that evaluate fitness-for-purpose and identify their analyses’ strengths and weaknesses. Within a defined cohort, reporting performance metrics by taxon, rather than by sample, will clarify this evaluation.Results: For a complete assessment, estimated limits of detection, positive and negative control samples, and true positive and negative true results are necessary criteria for all investigated taxa. Use of summary metrics should be restricted to comparing results of similar, or ideally the same, cohorts and data, and should employ harmonic means and continuous products for each performance metric rather than arithmetic mean. Conclusions: Organism-centric analysis and reporting will enable clear performance assessment and meaningful comparisons between methods in evaluating fitness for purpose of metagenomic analyses with their intended applications. |
