Artificial intelligence-based conversational agent to support medication prescribing
Autor: | Brett R. South, Mya Baca, Nawshin Kutub, Adam M. Buchold, Gretchen Purcell Jackson, Anita M. Preininger, Bryan S. Bohanan, Jeff Heiland, Rex Nipper, Suwei Wang |
---|---|
Rok vydání: | 2020 |
Předmět: |
Matching (statistics)
AcademicSubjects/SCI01060 Computer science Health Informatics Research and Applications computer.software_genre 03 medical and health sciences 0302 clinical medicine Medication.prescribing 030212 general & internal medicine natural language processing Dialog system Typographical error conversational agents business.industry 05 social sciences artificial intelligence pharmacological information systems Subject-matter expert machine learning Systems architecture Artificial intelligence AcademicSubjects/SCI01530 0509 other social sciences AcademicSubjects/MED00010 050904 information & library sciences business computer Natural language Scope (computer science) |
Zdroj: | JAMIA Open |
ISSN: | 2574-2531 |
Popis: | Objective This article describes the system architecture, training, initial use, and performance of Watson Assistant (WA), an artificial intelligence-based conversational agent, accessible within Micromedex®. Materials and methods The number and frequency of intents (target of a user’s query) triggered in WA during its initial use were examined; intents triggered over 9 months were compared to the frequency of topics accessed via keyword search of Micromedex. Accuracy of WA intents assigned to 400 queries was compared to assignments by 2 independent subject matter experts (SMEs), with inter-rater reliability measured by Cohen’s kappa. Results In over 126 000 conversations with WA, intents most frequently triggered involved dosing (N = 30 239, 23.9%) and administration (N = 14 520, 11.5%). SMEs with substantial inter-rater agreement (kappa = 0.71) agreed with intent mapping in 247 of 400 queries (62%), including 16 queries related to content that WA and SMEs agreed was unavailable in WA. SMEs found 57 (14%) of 400 queries incorrectly mapped by WA; 112 (28%) queries unanswerable by WA included queries that were either ambiguous, contained unrecognized typographical errors, or addressed topics unavailable to WA. Of the queries answerable by WA (288), SMEs determined 231 (80%) were correctly linked to an intent. Discussion A conversational agent successfully linked most queries to intents in Micromedex. Ongoing system training seeks to widen the scope of WA and improve matching capabilities. Conclusion WA enabled Micromedex users to obtain answers to many medication-related questions using natural language, with the conversational agent facilitating mapping to a broader distribution of topics than standard keyword searches. |
Databáze: | OpenAIRE |
Externí odkaz: |