Meta-context Transformers for Domain-Specific Response Generation
| Autor: | Amar Prakash Azad, Debanjana Kar, Suranjana Samanta |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
business.industry
Computer science 05 social sciences Context (language use) 010501 environmental sciences computer.software_genre 01 natural sciences Domain (software engineering) 0502 economics and business Key (cryptography) Relevance (information retrieval) Artificial intelligence Language model 050207 economics Performance improvement business computer Natural language processing Coherence (linguistics) 0105 earth and related environmental sciences Transformer (machine learning model) |
| Zdroj: | Advances in Knowledge Discovery and Data Mining ISBN: 9783030757670 PAKDD (3) |
| Popis: | Transformer-based models, such as GPT-2, have revolutionized the landscape of dialogue generation by capturing the long-range structures through language modeling. Though these models have exhibited excellent language coherence, they often lack relevance and terms when used for domain-specific response generation. In this paper, we present DSRNet (Domain Specific Response Network), a transformer-based model for dialogue response generation by reinforcing domain-specific attributes. In particular, we extract meta attributes from context and joinly model with the dialogue context utterances for better attention over domain-specific keyterms and relevance. We study the use of DSRNet in a multi-turn multi-interlocutor environment for domain-specific response generation. In our experiments, we evaluate DSRNet on Ubuntu dialogue datasets, which are mainly composed of various technical domain related dialogues for IT domain issue resolutions and also on CamRest676 dataset, which contains restaurant domain conversations. We observe that the responses produced by our model carry higher relevance due to the presence of domain-specific key attributes that exhibit better overlap with the attributes of the context. Our analysis shows that the performance improvement is mostly due to the infusion of key terms along with dialogues which result in better attention over domain-relevant terms. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|