Foundations of Machine Learning-Based Clinical Prediction Modeling: Part III-Model Evaluation and Other Points of Significance.
| Autor: | Staartjes VE; Machine Intelligence in Clinical Neuroscience (MICN) Laboratory, Department of Neurosurgery, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Zurich, Switzerland. victoregon.staartjes@usz.ch., Kernbach JM; Neurosurgical Artificial Intelligence Laboratory Aachen (NAILA), Department of Neurosurgery, RWTH Aachen University Hospital, Aachen, Germany. |
|---|---|
| Jazyk: | angličtina |
| Zdroj: | Acta neurochirurgica. Supplement [Acta Neurochir Suppl] 2022; Vol. 134, pp. 23-31. |
| DOI: | 10.1007/978-3-030-85292-4_4 |
| Abstrakt: | Various available metrics to describe model performance in terms of discrimination (area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value, negative predictive value, F1 Score) and calibration (slope, intercept, Brier score, expected/observed ratio, Estimated Calibration Index, Hosmer-Lemeshow goodness-of-fit) are presented. Recalibration is introduced, with Platt scaling and Isotonic regression as proposed methods. We also discuss considerations regarding the sample size required for optimal training of clinical prediction models-explaining why low sample sizes lead to unstable models, and offering the common rule of thumb of at least ten patients per class per input feature, as well as some more nuanced approaches. Missing data treatment and model-based imputation instead of mean, mode, or median imputation is also discussed. We explain how data standardization is important in pre-processing, and how it can be achieved using, e.g. centering and scaling. One-hot encoding is discussed-categorical features with more than two levels must be encoded as multiple features to avoid wrong assumptions. Regarding binary classification models, we discuss how to select a sensible predicted probability cutoff for binary classification using the closest-to-(0,1)-criterion based on AUC or based on the clinical question (rule-in or rule-out). Extrapolation is also discussed. (© 2022. The Author(s), under exclusive license to Springer Nature Switzerland AG.) |
| Databáze: | MEDLINE |
| Externí odkaz: |
|