Measurement of the Inter-Rater Reliability Rate Is Mandatory for Improving the Quality of a Medical Database: Experience with the Paulista Lung Cancer Registry
| Autor: | Leticia Leone Lauricella, Paulo Manuel Pêgo-Fernandes, Michele Salati, Ricardo Mingarini Terra, Priscila Berenice Costa |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Male
Lung Neoplasms Quality management Intraclass correlation 02 engineering and technology Audit 030204 cardiovascular system & hematology computer.software_genre 03 medical and health sciences Postoperative Complications 0302 clinical medicine Carcinoma Non-Small-Cell Lung 020204 information systems CENTROS DE REABILITAÇÃO 0202 electrical engineering electronic engineering information engineering Humans Medicine Longitudinal Studies Prospective Studies Registries Database business.industry Data Collection Standardized approach Reproducibility of Results Quality Improvement Confidence interval Inter-rater reliability Data quality Female Surgery business Completeness (statistics) computer Brazil |
| Zdroj: | Repositório Institucional da USP (Biblioteca Digital da Produção Intelectual) Universidade de São Paulo (USP) instacron:USP |
| Popis: | Background Database quality measurement should be considered a mandatory step to ensure an adequate level of confidence in data used for research and quality improvement. Several metrics have been described in the literature, but no standardized approach has been established. We aimed to describe a methodological approach applied to measure the quality and inter-rater reliability of a regional multicentric thoracic surgical database (Paulista Lung Cancer Registry). Study Design Data from the first 3 years of the Paulista Lung Cancer Registry underwent an audit process with 3 metrics: completeness, consistency, and inter-rater reliability. The first 2 methods were applied to the whole data set, and the last method was calculated using 100 cases randomized for direct auditing. Inter-rater reliability was evaluated using percentage of agreement between the data collector and auditor and through calculation of Cohen's κ and intraclass correlation. Results The overall completeness per section ranged from 0.88 to 1.00, and the overall consistency was 0.96. Inter-rater reliability showed many variables with high disagreement (>10%). For numerical variables, intraclass correlation was a better metric than inter-rater reliability. Cohen's κ showed that most variables had moderate to substantial agreement. Conclusions The methodological approach applied to the Paulista Lung Cancer Registry showed that completeness and consistency metrics did not sufficiently reflect the real quality status of a database. The inter-rater reliability associated with κ and intraclass correlation was a better quality metric than completeness and consistency metrics because it could determine the reliability of specific variables used in research or benchmark reports. This report can be a paradigm for future studies of data quality measurement. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|