Examining response process validity of script concordance testing: a think-aloud approach
| Autor: | Michael Siu Hong Wan, Elina Tor, Judith N Hudson |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
Adult
Male Examining Response Process Students Medical 020205 medical informatics Response process Concordance assessment Applied psychology education response process validity Script concordance testing 02 engineering and technology Clinical Reasoning Cohort Studies Thinking 03 medical and health sciences Young Adult 0302 clinical medicine Surveys and Questionnaires 0202 electrical engineering electronic engineering information engineering Humans Speech Student learning Set (psychology) Think aloud protocol Problem Solving Original Research Response rate (survey) Clinical reasoning Reproducibility of Results 030206 dentistry General Medicine Middle Aged Partial credit written think-aloud Female Clinical Competence Educational Measurement Psychology |
| Zdroj: | International Journal of Medical Education |
| ISSN: | 2042-6372 |
| Popis: | Objectives This study investigated whether medical student responses to Script Concordance Testing (SCT) items represent valid clinical reasoning. Using a think-aloud approach students provided written explanations of the reasoning that underpinned their responses, and these were reviewed for concordance with an expert reference panel. Methods A set of 12, 11 and 15 SCT items were administered online to Year 3 (2018), Year 4 (2018) and Year 3 (2019) medical students respectively. Students' free-text descriptions of the reasoning supporting each item response were analysed, and compared with those of the expert panel. Response process validity was quantified as the rate of true positives (percentage of full and partial credit responses derived through correct clinical reasoning); and true negatives (percentage of responses with no credit derived through faulty clinical reasoning). Results Two hundred and nine students completed the online tests (response rate = 68.3%). The majority of students who had chosen the response which attracted full or partial credit also provided justifications which were concordant with the experts (true positive rate of 99.6% for full credit; 99.4% for partial credit responses). Most responses that attracted no credit were based on faulty clinical reasoning (true negative of 99.0%). Conclusions The findings provide support for the response process validity of SCT scores in the setting of undergraduate medicine. The additional written think-aloud component, to assess clinical reasoning, provided useful information to inform student learning. However, SCT scores should be validated on each testing occasion, and in other contexts. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|