SparseVSR: Lightweight and Noise Robust Visual Speech Recognition

Autor:	Fernandez-Lopez, Adriana, Chen, Honglie, Ma, Pingchuan, Haliassos, Alexandros, Petridis, Stavros, Pantic, Maja
Rok vydání:	2023
Předmět:	Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Recent advances in deep neural networks have achieved unprecedented success in visual speech recognition. However, there remains substantial disparity between current methods and their deployment in resource-constrained devices. In this work, we explore different magnitude-based pruning techniques to generate a lightweight model that achieves higher performance than its dense model equivalent, especially under the presence of visual noise. Our sparse models achieve state-of-the-art results at 10% sparsity on the LRS3 dataset and outperform the dense equivalent up to 70% sparsity. We evaluate our 50% sparse model on 7 different visual noise types and achieve an overall absolute improvement of more than 2% WER compared to the dense equivalent. Our results confirm that sparse networks are more resistant to noise than dense networks. Comment: Accepted to Interspeech 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2307.04552 Zobrazit plný text záznamu View this record from Arxiv