Uncertainty-aware automatic TNM staging classification for [ 18 F] Fluorodeoxyglucose PET-CT reports for lung cancer utilising transformer-based language models and multi-task learning.

Autor: Barlow SH; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK. stephen.barlow@kcl.ac.uk., Chicklore S; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.; King's College London and Guy's and St. Thomas' PET Centre, St. Thomas' Hospital, London, UK., He Y; Department of Informatics, King's College London, London, UK.; Department of Computer Science, University of Warwick, Coventry, UK.; Alan Turing Institute, London, UK., Ourselin S; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK., Wagner T; Department of Nuclear Medicine, Royal Free Hospital, London, UK., Barnes A; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.; King's Technology Evaluation Centre (KiTEC), School of Biomedical Engineering & Imaging Science, King's College London, London, UK., Cook GJR; School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.; King's College London and Guy's and St. Thomas' PET Centre, St. Thomas' Hospital, London, UK.
Jazyk: angličtina
Zdroj: BMC medical informatics and decision making [BMC Med Inform Decis Mak] 2024 Dec 18; Vol. 24 (1), pp. 396. Date of Electronic Publication: 2024 Dec 18.
DOI: 10.1186/s12911-024-02814-7
Abstrakt: Background: [ 18 F] Fluorodeoxyglucose (FDG) PET-CT is a clinical imaging modality widely used in diagnosing and staging lung cancer. The clinical findings of PET-CT studies are contained within free text reports, which can currently only be categorised by experts manually reading them. Pre-trained transformer-based language models (PLMs) have shown success in extracting complex linguistic features from text. Accordingly, we developed a multi-task 'TNMu' classifier to classify the presence/absence of tumour, node, metastasis ('TNM') findings (as defined by The Eight Edition of TNM Staging for Lung Cancer). This is combined with an uncertainty classification task ('u') to account for studies with ambiguous TNM status.
Methods: 2498 reports were annotated by a nuclear medicine physician and split into train, validation, and test datasets. For additional evaluation an external dataset (n = 461 reports) was created, and annotated by two nuclear medicine physicians with agreement reached on all examples. We trained and evaluated eleven publicly available PLMs to determine which is most effective for PET-CT reports, and compared multi-task, single task and traditional machine learning approaches.
Results: We find that a multi-task approach with GatorTron as PLM achieves the best performance, with an overall accuracy (all four tasks correct) of 84% and a Hamming loss of 0.05 on the internal test dataset, and 79% and 0.07 on the external test dataset. Performance on the individual TNM tasks approached expert performance with macro average F1 scores of 0.91, 0.95 and 0.90 respectively on external data. For uncertainty an F1 of 0.77 is achieved.
Conclusions: Our 'TNMu' classifier successfully extracts TNM staging information from internal and external PET-CT reports. We concluded that multi-task approaches result in the best performance, and better computational efficiency over single task PLM approaches. We believe these models can improve PET-CT services by assisting in auditing, creating research cohorts, and developing decision support systems. Our approach to handling uncertainty represents a novel first step but has room for further refinement.
Competing Interests: Declarations. Ethics approval and consent to participate: The data use and collection was approved by United Kingdom Research Ethics Committee (UK IRAS 228790) as part of Guy’s Cancer Cohort (ref: 18/NW/0297) [43]. The Guy’s Cancer Cohort committee waived the need for individual consent to participate as the study uses retrospective, anonymised data. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests.
(© 2024. The Author(s).)
Databáze: MEDLINE
Externí odkaz: