Explaining the Attention Mechanism of End-to-End Speech Recognition Using Decision Trees

Autor:	Wang, Yuanchao, Du, Wenji, Cai, Chenghao, Xu, Yanyan
Rok vydání:	2021
Předmět:	Computer Science - Computation and Language Computer Science - Machine Learning Computer Science - Sound Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	The attention mechanism has largely improved the performance of end-to-end speech recognition systems. However, the underlying behaviours of attention is not yet clearer. In this study, we use decision trees to explain how the attention mechanism impact itself in speech recognition. The results indicate that attention levels are largely impacted by their previous states rather than the encoder and decoder patterns. Additionally, the default attention mechanism seems to put more weights on closer states, but behaves poorly on modelling long-term dependencies of attention states. Comment: 10 pages, 5 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2110.03879 Zobrazit plný text záznamu View this record from Arxiv