Is Automated Machine Translation sufficiently accurate to translate figurative language in a clinical setting? (Preprint)

Autor: Hailee Tougas, Steven Chan, Tara Shahrvini, Alvaro Gonzalez, Ruth Chun Reyes, Michelle Burke Parish, Peter Yellowlees
Rok vydání: 2022
DOI: 10.2196/preprints.39556
Popis: BACKGROUND Patients with Limited English Proficiency (LEP) frequently receive substandard healthcare. Asynchronous Telepsychiatry (ATP) has been established as a clinically valid method for psychiatric assessments (1). The addition of automated speech recognition (ASR) and automated machine translation (AMT) technologies to ATP may be a viable Artificial Intelligence (AI)-language interpretation option. OBJECTIVE This project measured the frequency and accuracy of translation of figurative language devices (FLDs), and the patient word count per minute in a subset of psychiatric interviews from a larger trial, as an approximation to patient speech complexity and quantity in clinical encounters requiring interpretation. METHODS Six patients were selected from the original trial, where they had undergone two assessments, once by an English-speaking psychiatrist through a Spanish-speaking human interpreter and once in Spanish by a trained mental health interviewer-researcher with AI-interpretation. Three patients were interviewed via videoconferencing because of the COVID-19 pandemic. Interview transcripts were created by ASR with manual corrections for transcriptional accuracy and assessment for translational accuracy of FLDs. RESULTS Both human and AI-interpreted FLDs were frequently translated inaccurately, while human-interpreted interviews were found to have a significant reduction in the use of FLDs and the patient word count per minute; FLD translation was more accurate on videoconferencing. CONCLUSIONS AI-interpretation is not sufficiently accurate at this time for use in clinical settings. However, this study suggests that alternatives to human interpretation are needed to circumvent modifications to patients’ speech. While AI-interpretation technologies are being further developed, using videoconferencing for human interpreting may be more accurate than in-person interpreting. CLINICALTRIAL Clinicaltrials.gov NCT03538860; https://clinicaltrials.gov/ct2/show/NCT03538860
Databáze: OpenAIRE