| Popis: |
BACKGROUND Field notes, a form for resident-preceptor clinical encounter feedback, are widely adopted across Canadian medical residency training programs for documenting resident performance. This process generates a sizeable cumulative collection of feedback text, which is difficult for medical education faculty to navigate. As sentiment analysis is a sub-field of text mining that can efficiently synthesize the polarity of a text collection, sentiment analysis may serve as an innovative solution. OBJECTIVE To examine the utility and feasibility of sentiment analysis using three popular sentiment lexicons on medical resident field notes. METHODS We used a retrospective cohort design, curating text data from University of Toronto medical resident field notes gathered over two years (July 2019 – June 2021). Lexicon-based sentiment analysis was applied using three standardized dictionaries, modified by removing ambiguous words as determined by a medical subject matter expert. Our modified lexicons assigned words from the text data a sentiment score, and we aggregated word-level scores to a document-level polarity score. Agreement between dictionaries was assessed, and the document-level polarity was correlated with the overall preceptor rating of the clinical encounter under assessment. RESULTS Across the three original dictionaries, approximately a third of labeled words in our field note corpus were deemed ambiguous and removed to create modified dictionaries. Across the three modified dictionaries, the mean sentiment for the field note “Strengths” section was mildly positive, and slightly less positive in the “Areas of Improvement” section. We observed reasonable agreement between dictionaries for sentiment scores in both field note sections. Overall, the proportion of positively-labeled documents increased with the overall preceptor rating, and the proportion of negatively-labeled documents decreased with the overall preceptor rating. CONCLUSIONS Applying sentiment analysis to analyze field notes systematically is feasible. However, the applicability of existing lexicons is limited in the medical setting, even after removing ambiguous words. Limited applicability warrants the need to generate new dictionaries specific to the medical education context. Additionally, aspect-based sentiment analysis may be applied to navigate the more nuanced structure of texts when identifying sentiments. Ultimately, this will allow for more robust inferences to discover opportunities for improving resident teaching curriculums. CLINICALTRIAL |
