How to Evaluate ASR Output for Named Entity Recognition?
| Autor: | Sophie Rosset, Olivier Galibert, Martine Adda-Decker, Mohamed Ameur Ben Jannet |
|---|---|
| Přispěvatelé: | Laboratoire National de Métrologie et d'Essais [Trappes] (LNE ), LPP - Laboratoire de Phonétique et Phonologie - UMR 7018 (LPP), Université Sorbonne Nouvelle - Paris 3-Centre National de la Recherche Scientifique (CNRS), Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur (LIMSI), Université Paris Saclay (COmUE)-Centre National de la Recherche Scientifique (CNRS)-Sorbonne Université - UFR d'Ingénierie (UFR 919), Sorbonne Université (SU)-Sorbonne Université (SU)-Université Paris-Saclay-Université Paris-Sud - Paris 11 (UP11) |
| Jazyk: | angličtina |
| Rok vydání: | 2015 |
| Předmět: |
Computer science
Speech recognition Word error rate 020206 networking & telecommunications Statistical model 02 engineering and technology Speech processing computer.software_genre 030507 speech-language pathology & audiology 03 medical and health sciences Named-entity recognition Transcription (linguistics) Speech translation 0202 electrical engineering electronic engineering information engineering Automatic speech [SHS.LANGUE]Humanities and Social Sciences/Linguistics 0305 other medical science computer |
| Zdroj: | 16th Annual Conference of the International Speech Communication Association (Interspeech'15) 16th Annual Conference of the International Speech Communication Association (Interspeech'15), Sep 2015, Dresden, Germany INTERSPEECH HAL |
| Popis: | International audience; The standard metric to evaluate automatic speech recognition (ASR) systems is the word error rate (WER). WER has proven very useful in stand-alone ASR systems. Nowadays, these systems are often embedded in complex natural language processing systems to perform tasks like speech translation, manmachine dialogue, or information retrieval from speech. This exacerbates the need for the speech processing community to design a new evaluation metric to estimate the quality of automatic transcriptions within their larger applicative context. We introduce a new measure to evaluate ASR in the context of named entity recognition, which makes use of a probabilistic model to estimate the risk of ASR errors inducing downstream errors in named entity detection. Our evaluation, on the ETAPE data, shows that ATENE achieves a higher correlation than WER between the performances in named entities recognition and in automatic speech transcription. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|