Application of Fusion of Various Spontaneous Speech Analytics Methods for Improving Far-Field Neural-Based Diarization
Autor: | Yuri Matveev, Sergei Astapov, Vladimir Kabarov, Vlada Kapranova, Marina Volkova, Elena Timofeeva, Aleksei Gusev, Valeriia Zaluskaia, Elena Evseeva, Aleksei Logunov |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2021 |
Předmět: |
Voice activity detection
Artificial neural network business.industry Computer science General Mathematics Speech recognition speaker number estimation speaker diarization spontaneous speech processing voice activity detection overlapping speech detection speaker extractor models model fusion quality estimation distant speech processing artificial neural networks Speaker diarisation Analytics Computer Science (miscellaneous) QA1-939 Speech analytics Noise (video) Transcription (software) Cluster analysis business Engineering (miscellaneous) Mathematics |
Zdroj: | Mathematics; Volume 9; Issue 23; Pages: 2998 Mathematics, Vol 9, Iss 2998, p 2998 (2021) |
ISSN: | 2227-7390 |
DOI: | 10.3390/math9232998 |
Popis: | Recently developed methods in spontaneous speech analytics require the use of speaker separation based on audio data, referred to as diarization. It is applied to widespread use cases, such as meeting transcription based on recordings from distant microphones and the extraction of the target speaker’s voice profiles from noisy audio. However, speech recognition and analysis can be hindered by background and point-source noise, overlapping speech, and reverberation, which all affect diarization quality in conjunction with each other. To compensate for the impact of these factors, there are a variety of supportive speech analytics methods, such as quality assessments in terms of SNR and RT60 reverberation time metrics, overlapping speech detection, instant speaker number estimation, etc. The improvements in speaker verification methods have benefits in the area of speaker separation as well. This paper introduces several approaches aimed towards improving diarization system quality. The presented experimental results demonstrate the possibility of refining initial speaker labels from neural-based VAD data by means of fusion with labels from quality estimation models, overlapping speech detectors, and speaker number estimation models, which contain CNN and LSTM modules. Such fusing approaches allow us to significantly decrease DER values compared to standalone VAD methods. Cases of ideal VAD labeling are utilized to show the positive impact of ResNet-101 neural networks on diarization quality in comparison with basic x-vectors and ECAPA-TDNN architectures trained on 8 kHz data. Moreover, this paper highlights the advantage of spectral clustering over other clustering methods applied to diarization. The overall quality of diarization is improved at all stages of the pipeline, and the combination of various speech analytics methods makes a significant contribution to the improvement of diarization quality. |
Databáze: | OpenAIRE |
Externí odkaz: | |
Nepřihlášeným uživatelům se plný text nezobrazuje | K zobrazení výsledku je třeba se přihlásit. |