Applying Object Detection and Large Language Model to Establish a Smart Telemedicine Diagnosis System with Chatbot: A Case Study of Pressure Injuries Diagnosis System.

Autor: Chen CC; Institute of Medicine, Chung Shan Medical University, Taichung, Taiwan, ROC.; Department of Plastic Surgery, Chi Mei Medical Center, Tainan, Taiwan, ROC., Wei CJ; Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan, ROC., Tseng TY; Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan, ROC., Chiu MC; Department of Industrial Engineering and Engineering Management, National Tsing Hua University, Hsinchu, Taiwan, ROC., Chang CC; Department of Medical Informatics, Chung Shan Medical University Hospital, Taichung, Taiwan, ROC.; IT Office, Chung Shan Medical University Hospital, Taichung, Taiwan, ROC.; Department of Information Management, Ming Chuan University, Taoyuan, Taiwan, ROC.
Jazyk: angličtina
Zdroj: Telemedicine journal and e-health : the official journal of the American Telemedicine Association [Telemed J E Health] 2024 Jun; Vol. 30 (6), pp. e1705-e1712. Date of Electronic Publication: 2024 Mar 21.
DOI: 10.1089/tmj.2023.0715
Abstrakt: Background: The scarcity of medical resources and personnel has worsened due to COVID-19. Telemedicine faces challenges in assessing wounds without physical examination. Evaluating pressure injuries is time consuming, energy intensive, and inconsistent. Most of today's telemedicine platforms utilize graphical user interfaces with complex operational procedures and limited channels for information dissemination. The study aims to establish a smart telemedicine diagnosis system based on YOLOv7 and large language model. Methods: The YOLOv7 model is trained using a clinical data set, with data augmentation techniques employed to enhance the data set to identify six types of pressure injury images. The established system features a front-end interface that includes responsive web design and a chatbot with ChatGPT, and it is integrated with a database for personal information management. Results: This research provides a practical pressure injury staging classification model with an average F1 score of 0.9238. The system remotely provides real-time accurate diagnoses and prescriptions, guiding patients to seek various medical help levels based on symptom severity. Conclusions: This study establishes a smart telemedicine auxiliary diagnosis system based on the YOLOv7 model, which possesses capabilities for classification and real-time detection. During teleconsultations, it provides immediate and accurate diagnostic information and prescription recommendations and seeks various medical assistance based on the severity of symptoms. Through the setup of a chatbot with ChatGPT, different users can quickly achieve their respective objectives.
Databáze: MEDLINE