Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation

Autor:	Thakker, Manthan, Eskimez, Sefik Emre, Yoshioka, Takuya, Wang, Huaming
Rok vydání:	2022
Předmět:	Electrical Engineering and Systems Science - Audio and Speech Processing Computer Science - Sound Electrical Engineering and Systems Science - Signal Processing
Druh dokumentu:	Working Paper
Popis:	This paper investigates how to improve the runtime speed of personalized speech enhancement (PSE) networks while maintaining the model quality. Our approach includes two aspects: architecture and knowledge distillation (KD). We propose an end-to-end enhancement (E3Net) model architecture, which is $3\times$ faster than a baseline STFT-based model. Besides, we use KD techniques to develop compressed student models without significantly degrading quality. In addition, we investigate using noisy data without reference clean signals for training the student models, where we combine KD with multi-task learning (MTL) using automatic speech recognition (ASR) loss. Our results show that E3Net provides better speech and transcription quality with a lower target speaker over-suppression (TSOS) rate than the baseline model. Furthermore, we show that the KD methods can yield student models that are $2-4\times$ faster than the teacher and provides reasonable quality. Combining KD and MTL improves the ASR and TSOS metrics without degrading the speech quality. Comment: Submitted to Interspeech conference 2022 https://interspeech2022.org/
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2204.00771 Zobrazit plný text záznamu View this record from Arxiv