Retrospective comparison of approaches to evaluating inter-observer variability in CT tumour measurements in an academic health centre
| Autor: | Ronald W. Gimbel, Steven C. Lowe, A. Michael Devane, Moonseong Heo, MinJae Woo |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
medicine.medical_specialty
protocols & guidelines Intraclass correlation Health centre quality in health care Consistency (statistics) medicine Humans Retrospective Studies Observer Variation business.industry Radiology and Imaging Liver Neoplasms Reproducibility of Results computed tomography General Medicine Response Evaluation Criteria in Solid Tumors adult oncology Outlier Medicine Pairwise comparison Radiology Tomography X-Ray Computed Observer variation No detection business |
| Zdroj: | BMJ Open, Vol 10, Iss 11 (2020) BMJ Open |
| ISSN: | 2044-6055 |
| Popis: | BackgroundA growing number of research studies have reported inter-observer variability in sizes of tumours measured from CT scans. It remains unclear whether the conventional statistical measures correctly evaluate the CT measurement consistency for optimal treatment management and decision-making. We compared and evaluated the existing measures for evaluating inter-observer variability in CT measurement of cancer lesions.Methods13 board-certified radiologists repeatedly reviewed 10 CT image sets of lung lesions and hepatic metastases selected through a randomisation process. A total of 130 measurements under RECIST 1.1 (Response Evaluation Criteria in Solid Tumors) guidelines were collected for the demonstration. Intraclass correlation coefficient (ICC), Bland-Altman plotting and outlier counting methods were selected for the comparison. The each selected measure was used to evaluate three cases with observed, increased and decreased inter-observer variability.ResultsThe ICC score yielded a weak detection when evaluating different levels of the inter-observer variability among radiologists (increased: 0.912; observed: 0.962; decreased: 0.990). The outlier counting method using Bland-Altman plotting with 2SD yielded no detection at all with its number of outliers unchanging regardless of level of inter-observer variability. Outlier counting based on domain knowledge was more sensitised to different levels of the inter-observer variability compared with the conventional measures (increased: 0.756; observed: 0.923; improved: 1.000). Visualisation of pairwise Bland-Altman bias was also sensitised to the inter-observer variability with its pattern rapidly changing in response to different levels of the inter-observer variability.ConclusionsConventional measures may yield weak or no detection when evaluating different levels of the inter-observer variability among radiologists. We observed that the outlier counting based on domain knowledge was sensitised to the inter-observer variability in CT measurement of cancer lesions. Our study demonstrated that, under certain circumstances, the use of standard statistical correlation coefficients may be misleading and result in a sense of false security related to the consistency of measurement for optimal treatment management and decision-making. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|