ForceReader: a BERT-based Interactive Machine Reading Comprehension Model with Attention Separation
| Autor: | Kangjian Wu, Zheng Chen |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Computer science
business.industry media_common.quotation_subject 05 social sciences Context (language use) 010501 environmental sciences computer.software_genre 01 natural sciences Ranking (information retrieval) Comprehension Reading comprehension Argument Reading (process) 0502 economics and business Input method Artificial intelligence 050207 economics Paragraph business computer Natural language processing 0105 earth and related environmental sciences media_common |
| Zdroj: | COLING |
| DOI: | 10.18653/v1/2020.coling-main.241 |
| Popis: | The release of BERT revolutionized the development of NLP. Various BERT-based reading comprehension models have been proposed, thus updating the performance ranking of reading comprehension tasks. However, the above BERT-based models inherently employ BERT’s combined input method, representing the input question and paragraph as a single packed sequence, without further modification for reading comprehension. This paper makes an in-depth analysis of this input method, proposes a problem of this approach. We call it attention deconcentration. Accordingly, this paper proposes ForceReader, a BERT-based interactive machine reading comprehension model. First, ForceReader proposes a novel solution called the Attention Separation Representation to respond to attention deconcentration. Moreover, starting from the logical nature of reading comprehension tasks, ForceReader adopts Multi-mode Reading and Interactive Reasoning strategy. For the calculation of attention, ForceReader employs Conditional Background Attention to solve the lack of the overall context semantic after the separation of attention. As an integral model, ForceReader shows a significant improvement in reading comprehension tasks compared to BERT. Moreover, this paper makes detailed visual analyses of the attention and propose strategies accordingly. This may be another argument to the explanations of the attention. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|