Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering
| Autor: | Nachane, Saeel Sandeep, Gramopadhye, Ojas, Chanda, Prateek, Ramakrishnan, Ganesh, Jadhav, Kshitij Sharad, Nandwani, Yatin, Raghu, Dinesh, Joshi, Sachindra |
|---|---|
| Rok vydání: | 2024 |
| Předmět: | |
| Druh dokumentu: | Working Paper |
| Popis: | In this paper, we propose a modified version of the MedQA-USMLE dataset, named MEDQA-OPEN, which contains open-ended medical questions without options to mimic clinical scenarios, along with clinician-approved reasoned answers. Additionally, we implement a prompt driven by Chain of Thought (CoT) reasoning, CLINICR, to mirror the prospective process of incremental reasoning, reaching a correct response to medical questions. We empirically demonstrate how CLINICR outperforms the state-of-the-art 5-shot CoT-based prompt (Li\'evin et al., 2022). We also present an approach that mirrors real-life clinical practice by first exploring multiple differential diagnoses through MCQ-CLINICR and subsequently narrowing down to a final diagnosis using MCQ-ELIMINATIVE. Finally, emphasizing the importance of response verification in medical settings, we utilize a reward model mechanism, replacing the elimination process performed by MCQ-ELIMINATIVE. Comment: The paper is accepted in EMNLP 2024 Findings |
| Databáze: | arXiv |
| Externí odkaz: |
|