OVERLAP-AWARE LOW-LATENCY ONLINE SPEAKER DIARIZATION BASED ON END-TO-END LOCAL SEGMENTATION
| Autor: | Juan Manuel Coria, Hervé Bredin, Sahar Ghannay, Sophie Rosset |
|---|---|
| Přispěvatelé: | Information, Langue Ecrite et Signée (ILES), Laboratoire Interdisciplinaire des Sciences du Numérique (LISN), CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Sciences et Technologies des Langues (STL), CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS), ANR-16-CE92-0025,PLUMCOT,Identification non-supervisée des personnages de films et séries télévisées(2016), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Sciences et Technologies des Langues (STL), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS) |
| Jazyk: | angličtina |
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Sound (cs.SD) overlapped speech detection [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE] Computer Science - Sound [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL] [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] [INFO.INFO-TT]Computer Science [cs]/Document and Text Processing low latency [INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing Audio and Speech Processing (eess.AS) FOS: Electrical engineering electronic engineering information engineering speaker embedding speaker diarization [INFO]Computer Science [cs] Electrical Engineering and Systems Science - Audio and Speech Processing |
| Zdroj: | IEEE Automatic Speech Recognition and Unserstanding Workshop IEEE Automatic Speech Recognition and Unserstanding Workshop, Dec 2021, Cartagena, Colombia HAL |
| Popis: | We propose to address online speaker diarization as a combination of incremental clustering and local diarization applied to a rolling buffer updated every 500ms. Every single step of the proposed pipeline is designed to take full advantage of the strong ability of a recently proposed end-to-end overlap-aware segmentation to detect and separate overlapping speakers. In particular, we propose a modified version of the statistics pooling layer (initially introduced in the x-vector architecture) to give less weight to frames where the segmentation model predicts simultaneous speakers. Furthermore, we derive cannot-link constraints from the initial segmentation step to prevent two local speakers from being wrongfully merged during the incremental clustering step. Finally, we show how the latency of the proposed approach can be adjusted between 500ms and 5s to match the requirements of a particular use case, and we provide a systematic analysis of the influence of latency on the overall performance (on AMI, DIHARD and VoxConverse). Comment: To appear in ASRU 2021. Code available at https://github.com/juanmc2005/StreamingSpeakerDiarization/ |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|