Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning
| Autor: | Shakeel A. Sheikh, Md Sahidullah, Fabrice Hirsch, Slim Ouni |
|---|---|
| Přispěvatelé: | Speech Modeling for Facilitating Oral-Based Communication (MULTISPEECH), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Department of Natural Language Processing & Knowledge Discovery (LORIA - NLPKD), Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS), Praxiling (Praxiling), Université Paul-Valéry - Montpellier 3 (UPVM)-Centre National de la Recherche Scientifique (CNRS) |
| Jazyk: | angličtina |
| Rok vydání: | 2023 |
| Předmět: |
FOS: Computer and information sciences
Sound (cs.SD) Computer Science - Machine Learning Data augmentation Stuttering detection Health Informatics Class balanced learning Computer Science - Sound Computer Science Applications Machine Learning (cs.LG) [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI] Health Information Management [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] Audio and Speech Processing (eess.AS) FOS: Electrical engineering electronic engineering information engineering Electrical and Electronic Engineering Speech disorder Electrical Engineering and Systems Science - Audio and Speech Processing |
| Zdroj: | IEEE Journal of Biomedical and Health Informatics IEEE Journal of Biomedical and Health Informatics, 2023, ⟨10.1109/JBHI.2023.3248281⟩ |
| ISSN: | 2168-2194 |
| Popis: | Stuttering is a neuro-developmental speech impairment characterized by uncontrolled utterances (interjections) and core behaviors (blocks, repetitions, and prolongations), and is caused by the failure of speech sensorimotors. Due to its complex nature, stuttering detection (SD) is a difficult task. If detected at an early stage, it could facilitate speech therapists to observe and rectify the speech patterns of persons who stutter (PWS). The stuttered speech of PWS is usually available in limited amounts and is highly imbalanced. To this end, we address the class imbalance problem in the SD domain via a multibranching (MB) scheme and by weighting the contribution of classes in the overall loss function, resulting in a huge improvement in stuttering classes on the SEP-28k dataset over the baseline (StutterNet). To tackle data scarcity, we investigate the effectiveness of data augmentation on top of a multi-branched training scheme. The augmented training outperforms the MB StutterNet (clean) by a relative margin of 4.18% in macro F1-score (F1). In addition, we propose a multi-contextual (MC) StutterNet, which exploits different contexts of the stuttered speech, resulting in an overall improvement of 4.48% in F 1 over the single context based MB StutterNet. Finally, we have shown that applying data augmentation in the cross-corpora scenario can improve the overall SD performance by a relative margin of 13.23% in F1 over the clean training. Accepted in IEEE Journal of Biomedical Health Informatics 2023 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|