Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback.

Autor: Gin BC; Department of Pediatrics, University of California San Francisco, 550 16th St Floor 4, UCSF Box 0110, San Francisco, CA, 94158, USA. brian.gin@ucsf.edu., Ten Cate O; Utrecht Center for Research and Development of Health Professions Education, University Medical Center, Utrecht, the Netherlands.; Department of Medicine, University of California San Francisco, San Francisco, USA., O'Sullivan PS; Department of Medicine, University of California San Francisco, San Francisco, USA.; Department of Surgery, University of California San Francisco, San Francisco, USA., Boscardin C; Department of Medicine, University of California San Francisco, San Francisco, USA.; Department of Anesthesia, University of California San Francisco, San Francisco, USA.
Jazyk: angličtina
Zdroj: Advances in health sciences education : theory and practice [Adv Health Sci Educ Theory Pract] 2024 Nov; Vol. 29 (5), pp. 1571-1592. Date of Electronic Publication: 2024 Feb 23.
DOI: 10.1007/s10459-024-10311-9
Abstrakt: The entrustment framework redirects assessment from considering only trainees' competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors' themes predominantly reflected skills related to patient presentations, while trainees' themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees' documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees' assimilation into their roles and formation of trusting relationships.
(© 2024. The Author(s).)
Databáze: MEDLINE
Externí odkaz: