| Popis: |
Background and Purpose. Evaluation of student performance during standardized patient (SP) experiences in physical therapist (PT) education has not been well studied. As the use of SP-based evaluation grows in PT education, it is important that programs take steps to develop appropriate passing standards to inform grading decisions. The Angoff method has been used to set passing standards in SP experiences in other health care professions and provides a process for setting credible passing standards for SP-based examination in PT education. The purpose of this study was to compare the passing standard (cut-scores) and passing rates set by 2 separate cohorts of judges using the Angoff method, and 2 traditionally used standard-setting methods during an SP-based assessment in early PT education. Another purpose was to evaluate the level of agreement between judges and the reliability of judges in both cohorts.Participants. Two cohorts of PT expert judges (8 academic faculty and 7 clinicians) participated in separate Angoff method standard-setting sessions. Thirty-eight first-year DPT students completed 3 separate SP cases.Methods. Passing cut-scores were determined by each cohort using the Angoff method. The reliability of each cut-score and passing rates were compared for each cohort and with 2 traditional standards (arbitrary 70% and 1 standard deviation [SD] below the mean).Outcomes. Cut-scores set using the Angoff method were lower and yielded higher passing rates than the traditional standards commonly used to make grading decisions. Cut-scores set using the Angoff method demonstrated significant agreement and reliability between judges in both cohorts.Discussion and Conclusions. This study highlights that different standard-setting methods produce different cut-scores and passing rates when applied to the same examination. Care should be taken to employ standards that rely on expert judge consensus in order to provide a rationale for grading and to appropriately measure student performance during high-stakes SP-based examinations.Key Words: Passing standards, Standardized patients, Angoff method, Student performance.BACKGROUND AND PURPOSEThe American Physical Therapy Association (APTA) has included the use of standardized patients (SPs) as part of its Educational Strategic Plan (2006-2020)} Currently, approximately one third of physical therapist (PT) education programs in the United States utilize standardized patients.2 Despite the growing use of SPs in PT education and the intent to use SP-based examination in the assessment of PT student competency, there is no literature documenting the processes used to determine expected passing standards on these exams in PT education. The purpose of this study was to explore the use of the Angoff method as a standard-setting process for use with SP exams in PT education.Standardized patients are individuals trained to portray a written case that is focused on a specific medical problem.3 Learners engage SPs in the same way as actual patients providing opportunities for the evaluation of competencies in the areas of interpersonal communication, professionalism, time management, physical examination skills, clinical reasoning, and overall clinical readiness to progress.4'8 Advantages of SPbased learning activities include the ability of faculty to tailor case studies appropriately for the level of learners and that a case can be portrayed consistently for all learners, creating a standardized performance assessment opportunity. To date, PT education programs appear to primarily use SPs in the formative evaluation of students in the areas of patient interviewing skills, the musculoskeletal examination, and treatment.2Student performance during an SP-based examination is typically evaluated using casespecific checklists.9,10 Case-specific checklists are developed by faculty and are composed of important items and behaviors that define the standard of care for a specific patient presentation. … |
