Annotating Norwegian Language Varieties on Twitter for Part-of-Speech

Autor:	Mæhlum, Petter, Kåsen, Andre, Touileb, Samia, Barnes, Jeremy
Rok vydání:	2022
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Norwegian Twitter data poses an interesting challenge for Natural Language Processing (NLP) tasks. These texts are difficult for models trained on standardized text in one of the two Norwegian written forms (Bokm{\aa}l and Nynorsk), as they contain both the typical variation of social media text, as well as a large amount of dialectal variety. In this paper we present a novel Norwegian Twitter dataset annotated with POS-tags. We show that models trained on Universal Dependency (UD) data perform worse when evaluated against this dataset, and that models trained on Bokm{\aa}l generally perform better than those trained on Nynorsk. We also see that performance on dialectal tweets is comparable to the written standards for some models. Finally we perform a detailed analysis of the errors that models commonly make on this data. Comment: Accepted at the Ninth Workshop on NLP for Similar Languages, Varieties and Dialects (Vardial2022). Collocated with COLING2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2210.06150 Zobrazit plný text záznamu View this record from Arxiv