Validity evidence for summative performance evaluations in postgraduate community pharmacy education.

Autor: Westein MPD; Department of Pharmaceutical Sciences, Utrecht University, Royal Dutch Pharmacists Association (KNMP), Research in Education, Faculty of Medicine Vrije Universiteit, Amsterdam, the Netherlands. Electronic address: m.p.d.westein@uu.nl., Koster AS; Department of Pharmaceutical Sciences, Utrecht University, Utrecht, the Netherlands. Electronic address: a.s.koster@uu.nl., Daelmans HEM; Master's programme of Medicine, Faculty of Medicine Vrije Universiteit, Amsterdam, the Netherlands. Electronic address: hem.daelmans@amsterdamumc.nl., Collares CF; Maastricht University Faculty of Health Medicine and Life Sciences, Maastricht, the Netherlands. Electronic address: c.collares@maastrichtuniversity.nl., Bouvy ML; Department of Pharmaceutical Sciences, Utrecht University, Utrecht, the Netherlands. Electronic address: m.l.bouvy@uu.nl., Kusurkar RA; Research in Education, Faculty of Medicine Vrije Universiteit, Amsterdam, the Netherlands. Electronic address: r.kusurkar@amsterdamumc.nl.
Jazyk: angličtina
Zdroj: Currents in pharmacy teaching & learning [Curr Pharm Teach Learn] 2022 Jun; Vol. 14 (6), pp. 701-711. Date of Electronic Publication: 2022 Jun 24.
DOI: 10.1016/j.cptl.2022.06.014
Abstrakt: Introduction: Workplace-based assessment of competencies is complex. In this study, the validity of summative performance evaluations (SPEs) made by supervisors in a two-year longitudinal supervisor-trainee relationship was investigated in a postgraduate community pharmacy specialization program in the Netherlands. The construct of competence was based on an adapted version of the 2005 Canadian Medical Education Directive for Specialists (CanMEDS) framework.
Methods: The study had a case study design. Both quantitative and qualitative data were collected. The year 1 and year 2 SPE scores of 342 trainees were analyzed using confirmatory factor analysis and generalizability theory. Semi-structured interviews were held with 15 supervisors and the program director to analyze the inferences they made and the impact of SPE scores on the decision-making process.
Results: A good model fit was found for the adapted CanMEDS based seven-factor construct. The reliability/precision of the SPE measurements could not be completely isolated, as every trainee was trained in one pharmacy and evaluated by one supervisor. Qualitative analysis revealed that supervisors varied in their standards for scoring competencies. Some supervisors were reluctant to fail trainees. The competency scores had little impact on the high-stakes decision made by the program director.
Conclusions: The adapted CanMEDS competency framework provided a valid structure to measure competence. The reliability/precision of SPE measurements could not be established and the SPE measurements provided limited input for the decision-making process. Indications of a shadow assessment system in the pharmacies need further investigation.
(Copyright © 2022 The Authors. Published by Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: