Retrieval Augmented Generation for Domain-specific Question Answering

Autor:	Sharma, Sanat, Yoon, David Seunghyun, Dernoncourt, Franck, Sultania, Dewang, Bagga, Karishma, Zhang, Mengjiao, Bui, Trung, Kotte, Varun
Rok vydání:	2024
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Information Retrieval Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Question answering (QA) has become an important application in the advanced development of large language models. General pre-trained large language models for question-answering are not trained to properly understand the knowledge or terminology for a specific domain, such as finance, healthcare, education, and customer service for a product. To better cater to domain-specific understanding, we build an in-house question-answering system for Adobe products. We propose a novel framework to compile a large question-answer database and develop the approach for retrieval-aware finetuning of a Large Language model. We showcase that fine-tuning the retriever leads to major improvements in the final generation. Our overall approach reduces hallucinations during generation while keeping in context the latest retrieval information for contextual grounding. Comment: AAAI 2024 (Association for the Advancement of Artificial Intelligence) Scientific Document Understanding Workshop
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2404.14760 Zobrazit plný text záznamu View this record from Arxiv