Inter-rater Agreement Between Exposure Assessment Using Automatic Algorithms and Using Experts
| Autor: | Marie-Élise Parent, Ines Florath, Mounia Senhaji Rhazi, Deborah Catherine Glass, Lin Fritschi |
|---|---|
| Přispěvatelé: | Curtin University [Perth], Planning and Transport Research Centre (PATREC), Monash University [Clayton], Institut Armand Frappier (INRS-IAF), Institut National de la Recherche Scientifique [Québec] (INRS)-Réseau International des Instituts Pasteur (RIIP), Funding for this project was provided by National Health and Medical Research Council (#1056684), SafeWork Australia, and the Cancer Research Society (Canada)., L.F. is supported by fellowships from the NHMRC and Cancer Council Western Australia. Marie-Elise Parent holds career awards from the Fonds de la recherche du Québec – Santé. We thank the other members of the AWES-Asthma team: Geza Benke, Julie Crewe, Renee Carey, Ellie Darcy, Sonia El-Zaemey, Michael Abramson, and Susan Peters. |
| Rok vydání: | 2018 |
| Předmět: |
Adult
Male Occupational group Future studies exposure assessment [SDV]Life Sciences [q-bio] 03 medical and health sciences Young Adult 0302 clinical medicine Cohen's kappa inter-rater reliability Occupational Exposure Humans Equal size expert judgement Exposure assessment Public Health Environmental and Occupational Health Allergens Middle Aged epidemiology methodology 030210 environmental & occupational health Asthma Research Personnel Inter-rater reliability Cross-Sectional Studies 030228 respiratory system Female Null hypothesis Psychology Algorithm Kappa Algorithms Environmental Monitoring |
| Zdroj: | Annals of Work Exposures and Health Annals of Work Exposures and Health, Oxford University Press, 2018, 63 (1), pp.45-53. ⟨10.1093/annweh/wxy084⟩ |
| ISSN: | 2398-7316 2398-7308 |
| DOI: | 10.1093/annweh/wxy084⟩ |
| Popis: | International audience; Objectives: To estimate the inter-rater agreement between exposure assessment to asthmagens in current jobs by algorithms based on task-based questionnaires (OccIDEAS) and by experts.Methods: Participants in a cross-sectional national survey of exposure to asthmagens (AWES-Asthma) were randomly split into two subcohorts of equal size. Subcohort 1 was used to determine the most common asthmagen groups and occupational groups. From subcohort 2, a random sample of 200 participants was drawn and current occupational exposure (yes/no) was assessed in these by OccIDEAS and by two experts independently and then as a consensus. Inter-rater agreement was estimated using Cohen's Kappa coefficient. The null hypothesis was set at 0.4, because both the experts and the automatic algorithm assessed the exposure using the same task-based questionnaires and therefore an agreement better than by chance would be expected.Results: The Kappa coefficients for the agreement between the experts and the algorithm-based assessments ranged from 0.37 to 1, while the agreement between the two experts ranged from 0.29 to 0.94, depending on the agent being assessed. After discussion by both experts the Kappa coefficients for the consensus decision and OccIDEAS were significantly larger than 0.4 for 7 of the 10 asthmagen groups, while overall the inter-rater agreement was greater than by chance (P < 0.0001).Conclusions: The web-based application OccIDEAS is an appropriate tool for automated assessment of current exposure to asthmagens (yes/no), and requires less time-consuming work by highly-qualified research personnel than the traditional expert-based method. Further, it can learn and reuse expert determinations in future studies. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|