Effect of Training Data Volume on Performance of Convolutional Neural Network Pneumothorax Classifiers.

Autor: Thian YL; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore. yee_liang_thian@nuhs.edu.sg., Ng DW; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore.; Saw Swee Hock School of Public Health, School of Computer Science, Yong Loo Lin School of Medicine, National University of Singapore, 12 Science Drive 2, #10-01, Queenstown, 117549, Singapore., Hallinan JTPD; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore., Jagmohan P; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore., Sia SY; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore., Mohamed JSA; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore.; Salmaniya Medical Complex Rd 2904, Manama, Bahrain., Quek ST; Department of Diagnostic Imaging, National University Hospital, 5 Lower Kent Ridge Rd, Queenstown, 119074, Singapore., Feng M; Saw Swee Hock School of Public Health, School of Computer Science, Yong Loo Lin School of Medicine, National University of Singapore, 12 Science Drive 2, #10-01, Queenstown, 117549, Singapore.
Jazyk: angličtina
Zdroj: Journal of digital imaging [J Digit Imaging] 2022 Aug; Vol. 35 (4), pp. 881-892. Date of Electronic Publication: 2022 Mar 03.
DOI: 10.1007/s10278-022-00594-y
Abstrakt: Large datasets with high-quality labels required to train deep neural networks are challenging to obtain in the radiology domain. This work investigates the effect of training dataset size on the performance of deep learning classifiers, focusing on chest radiograph pneumothorax detection as a proxy visual task in the radiology domain. Two open-source datasets (ChestX-ray14 and CheXpert) comprising 291,454 images were merged and convolutional neural networks trained with stepwise increase in training dataset sizes. Model iterations at each dataset volume were evaluated on an external test set of 525 emergency department chest radiographs. Learning curve analysis was performed to fit the observed AUCs for all models generated. For all three network architectures tested, model AUCs and accuracy increased rapidly from 2 × 10 3 to 20 × 10 3 training samples, with more gradual increase until the maximum training dataset size of 291 × 10 3 images. AUCs for models trained with the maximum tested dataset size of 291 × 10 3 images were significantly higher than models trained with 20 × 10 3 images: ResNet-50: AUC 20k  = 0.86, AUC 291k  = 0.95, p < 0.001; DenseNet-121 AUC 20k  = 0.85, AUC 291k  = 0.93, p < 0.001; EfficientNet AUC 20k  = 0.92, AUC 291 k  = 0.98, p < 0.001. Our study established learning curves describing the relationship between dataset training size and model performance of deep learning convolutional neural networks applied to a typical radiology binary classification task. These curves suggest a point of diminishing performance returns for increasing training data volumes, which algorithm developers should consider given the high costs of obtaining and labelling radiology data.
(© 2022. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)
Databáze: MEDLINE
Externí odkaz: