Probing Statistical Representations For End-To-End ASR

Autor:	Ollerenshaw, Anna, Jalal, Md Asif, Hain, Thomas
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language Computer Science - Sound Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	End-to-End automatic speech recognition (ASR) models aim to learn a generalised speech representation to perform recognition. In this domain there is little research to analyse internal representation dependencies and their relationship to modelling approaches. This paper investigates cross-domain language model dependencies within transformer architectures using SVCCA and uses these insights to exploit modelling approaches. It was found that specific neural representations within the transformer layers exhibit correlated behaviour which impacts recognition performance. Altogether, this work provides analysis of the modelling approaches affecting contextual dependencies and ASR performance, and can be used to create or adapt better performing End-to-End ASR models and also for downstream tasks. Comment: Submitted to ICASSP 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2211.01993 Zobrazit plný text záznamu View this record from Arxiv