Attention-based Conditioning Methods for External Knowledge Integration
| Autor: | Katerina Margatina, Christos Baziotis, Alexandros Potamianos |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer science Machine Learning (stat.ML) 02 engineering and technology 010501 environmental sciences Machine learning computer.software_genre Lexicon 01 natural sciences Machine Learning (cs.LG) Knowledge integration Statistics - Machine Learning 0202 electrical engineering electronic engineering information engineering 0105 earth and related environmental sciences Computer Science - Computation and Language business.industry Recurrent neural network Feature (computer vision) Benchmark (computing) 020201 artificial intelligence & image processing Affine transformation Artificial intelligence business computer Computation and Language (cs.CL) |
| Zdroj: | ACL (1) |
| Popis: | In this paper, we present a novel approach for incorporating external knowledge in Recurrent Neural Networks (RNNs). We propose the integration of lexicon features into the self-attention mechanism of RNN-based architectures. This form of conditioning on the attention distribution, enforces the contribution of the most salient words for the task at hand. We introduce three methods, namely attentional concatenation, feature-based gating and affine transformation. Experiments on six benchmark datasets show the effectiveness of our methods. Attentional feature-based gating yields consistent performance improvement across tasks. Our approach is implemented as a simple add-on module for RNN-based models with minimal computational overhead and can be adapted to any deep neural architecture. ACL 2019 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|