Examining a sentiment algorithm on session patient records in an eating disorder treatment setting: a preliminary study.

Autor:	Huisman SM; Department of Psychology, Health and Technology, Centre for eHealth and Wellbeing Research, University of Twente, Enschede, Netherlands., Kraiss JT; Department of Psychology, Health and Technology, Centre for eHealth and Wellbeing Research, University of Twente, Enschede, Netherlands., de Vos JA; Department of Research, GGZ Friesland Mental Healthcare Institution, Leeuwarden, Netherlands.; Human Concern, Centrum Voor Eetstoornissen, Amsterdam, Netherlands.
Jazyk:	angličtina
Zdroj:	Frontiers in psychiatry [Front Psychiatry] 2024 Mar 13; Vol. 15, pp. 1275236. Date of Electronic Publication: 2024 Mar 13 (Print Publication: 2024).
DOI:	10.3389/fpsyt.2024.1275236
Abstrakt:	Background: Clinicians collect session therapy notes within patient session records. Session records contain valuable information about patients' treatment progress. Sentiment analysis is a tool to extract emotional tones and states from text input and could be used to evaluate patients' sentiment during treatment over time. This preliminary study aims to investigate the validity of automated sentiment analysis on session patient records within an eating disorder (ED) treatment context against the performance of human raters. Methods: A total of 460 patient session records from eight participants diagnosed with an ED were evaluated on their overall sentiment by an automated sentiment analysis and two human raters separately. The inter-rater agreement (IRR) between the automated analysis and human raters and IRR among the human raters was analyzed by calculating the intra-class correlation (ICC) under a continuous interpretation and weighted Cohen's kappa under a categorical interpretation. Furthermore, differences regarding positive and negative matches between the human raters and the automated analysis were examined in closer detail. Results: The ICC showed a moderate automated-human agreement (ICC = 0.55), and the weighted Cohen's kappa showed a fair automated-human (k = 0.29) and substantial human-human agreement (k = 0.68) for the evaluation of overall sentiment. Furthermore, the automated analysis lacked words specific to an ED context. Discussion/conclusion: The automated sentiment analysis performed worse in discerning sentiment from session patient records compared to human raters and cannot be used within practice in its current state if the benchmark is considered adequate enough. Nevertheless, the automated sentiment analysis does show potential in extracting sentiment from session records. The automated analysis should be further developed by including context-specific ED words, and a more solid benchmark, such as patients' own mood, should be established to compare the performance of the automated analysis to. Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. (Copyright © 2024 Huisman, Kraiss and de Vos.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu