External validation is necessary in prediction research: a clinical example.

Autor: Bleeker SE; Erasmus Medical Center/Sophia Children's Hospital Department of Pediatrics, Room Sp 1545 Dr Molewaterplein 60, 3015 GJ Rotterdam, The Netherlands. sebleeker@yahoo.com, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, Moons KG
Jazyk: angličtina
Zdroj: Journal of clinical epidemiology [J Clin Epidemiol] 2003 Sep; Vol. 56 (9), pp. 826-32.
DOI: 10.1016/s0895-4356(03)00207-5
Abstrakt: Background and Objectives: Prediction models tend to perform better on data on which the model was constructed than on new data. This difference in performance is an indication of the optimism in the apparent performance in the derivation set. For internal model validation, bootstrapping methods are recommended to provide bias-corrected estimates of model performance. Results are often accepted without sufficient regard to the importance of external validation. This report illustrates the limitations of internal validation to determine generalizability of a diagnostic prediction model to future settings.
Methods: A prediction model for the presence of serious bacterial infections in children with fever without source was derived and validated internally using bootstrap resampling techniques. Subsequently, the model was validated externally.
Results: In the derivation set (n=376), nine predictors were identified. The apparent area under the receiver operating characteristic curve (95% confidence interval) of the model was 0.83 (0.78-0.87) and 0.76 (0.67-0.85) after bootstrap correction. In the validation set (n=179) the performance was 0.57 (0.47-0.67).
Conclusion: For relatively small data sets, internal validation of prediction models by bootstrap techniques may not be sufficient and indicative for the model's performance in future patients. External validation is essential before implementing prediction models in clinical practice.
Databáze: MEDLINE
Externí odkaz: