Zobrazeno 1 - 10
of 102
pro vyhledávání: '"Cieliebak, Mark"'
Autor:
Michot, Janick, Hürlimann, Manuela, Deriu, Jan, Sauer, Luzia, Mlynchyk, Katsiaryna, Cieliebak, Mark
One of the central skills that language learners need to practice is speaking the language. Currently, students in school do not get enough speaking opportunities and lack conversational practice. Recent advances in speech technology and natural lang
Externí odkaz:
http://arxiv.org/abs/2406.03235
Generative AI systems have become ubiquitous for all kinds of modalities, which makes the issue of the evaluation of such models more pressing. One popular approach is preference ratings, where the generated outputs of different systems are shown to
Externí odkaz:
http://arxiv.org/abs/2406.01131
Autor:
Paonessa, Claudio, Schraner, Yanick, Deriu, Jan, Hürlimann, Manuela, Vogel, Manfred, Cieliebak, Mark
This paper investigates the challenges in building Swiss German speech translation systems, specifically focusing on the impact of dialect diversity and differences between Swiss German and Standard German. Swiss German is a spoken language with no f
Externí odkaz:
http://arxiv.org/abs/2310.09088
A major challenge in the field of Text Generation is evaluation: Human evaluations are cost-intensive, and automated metrics often display considerable disagreement with human judgments. In this paper, we propose a statistical model of Text Generatio
Externí odkaz:
http://arxiv.org/abs/2306.03866
Autor:
Plüss, Michel, Deriu, Jan, Schraner, Yanick, Paonessa, Claudio, Hartmann, Julia, Schmidt, Larissa, Scheller, Christian, Hürlimann, Manuela, Samardžić, Tanja, Vogel, Manfred, Cieliebak, Mark
We present STT4SG-350 (Speech-to-Text for Swiss German), a corpus of Swiss German speech, annotated with Standard German text at the sentence level. The data is collected using a web app in which the speakers are shown Standard German sentences, whic
Externí odkaz:
http://arxiv.org/abs/2305.18855
Autor:
Belz, Anya, Thomson, Craig, Reiter, Ehud, Abercrombie, Gavin, Alonso-Moral, Jose M., Arvan, Mohammad, Braggaar, Anouck, Cieliebak, Mark, Clark, Elizabeth, van Deemter, Kees, Dinkar, Tanvi, Dušek, Ondřej, Eger, Steffen, Fang, Qixiang, Gao, Mingqi, Gatt, Albert, Gkatzia, Dimitra, González-Corbelle, Javier, Hovy, Dirk, Hürlimann, Manuela, Ito, Takumi, Kelleher, John D., Klubicka, Filip, Krahmer, Emiel, Lai, Huiyuan, van der Lee, Chris, Li, Yiru, Mahamood, Saad, Mieskes, Margot, van Miltenburg, Emiel, Mosteiro, Pablo, Nissim, Malvina, Parde, Natalie, Plátek, Ondřej, Rieser, Verena, Ruan, Jie, Tetreault, Joel, Toral, Antonio, Wan, Xiaojun, Wanner, Leo, Watson, Lewis, Yang, Diyi
We report our efforts in identifying a set of previous human evaluations in NLP that would be suitable for a coordinated study examining what makes human evaluations in NLP more/less reproducible. We present our results and findings, which include th
Externí odkaz:
http://arxiv.org/abs/2305.01633
A major challenge in the field of Text Generation is evaluation because we lack a sound theory that can be leveraged to extract guidelines for evaluation campaigns. In this work, we propose a first step towards such a theory that incorporates differe
Externí odkaz:
http://arxiv.org/abs/2210.13025
Autor:
Plüss, Michel, Hürlimann, Manuela, Cuny, Marc, Stöckli, Alla, Kapotis, Nikolaos, Hartmann, Julia, Ulasik, Malgorzata Anna, Scheller, Christian, Schraner, Yanick, Jain, Amit, Deriu, Jan, Cieliebak, Mark, Vogel, Manfred
We present SDS-200, a corpus of Swiss German dialectal speech with Standard German text translations, annotated with dialect, age, and gender information of the speakers. The dataset allows for training speech translation, dialect recognition, and sp
Externí odkaz:
http://arxiv.org/abs/2205.09501
This paper introduces an adversarial method to stress-test trained metrics to evaluate conversational dialogue systems. The method leverages Reinforcement Learning to find response strategies that elicit optimal scores from the trained metrics. We ap
Externí odkaz:
http://arxiv.org/abs/2202.13887
Spot The Bot: A Robust and Efficient Framework for the Evaluation of Conversational Dialogue Systems
Autor:
Deriu, Jan, Tuggener, Don, von Däniken, Pius, Campos, Jon Ander, Rodrigo, Alvaro, Belkacem, Thiziri, Soroa, Aitor, Agirre, Eneko, Cieliebak, Mark
The lack of time-efficient and reliable evaluation methods hamper the development of conversational dialogue systems (chatbots). Evaluations requiring humans to converse with chatbots are time and cost-intensive, put high cognitive demands on the hum
Externí odkaz:
http://arxiv.org/abs/2010.02140