Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers
| Autor: | Anna Salmela, Douglas Biber, Sampo Pyysalo, Veronika Laippala, Jesse Egbert, Liina Repo, Samuel Rönnqvist, Miika Oinonen, Valtteri Skantsi, Saara Hellström |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Register (sociolinguistics)
Cross lingual Computer science business.industry Shot (filmmaking) InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL Automatic processing computer.software_genre Zero (linguistics) Variation (linguistics) ComputingMethodologies_DOCUMENTANDTEXTPROCESSING Language model Artificial intelligence Affect (linguistics) business computer Natural language processing |
| Zdroj: | EACL (Student Research Workshop) |
| Popis: | We explore cross-lingual transfer of register classification for web documents. Registers, that is, text varieties such as blogs or news are one of the primary predictors of linguistic variation and thus affect the automatic processing of language. We introduce two new register-annotated corpora, FreCORE and SweCORE, for French and Swedish. We demonstrate that deep pre-trained language models perform strongly in these languages and outperform previous state-of-the-art in English and Finnish. Specifically, we show 1) that zero-shot cross-lingual transfer from the large English CORE corpus can match or surpass previously published monolingual models, and 2) that lightweight monolingual classification requiring very little training data can reach or surpass our zero-shot performance. We further analyse classification results finding that certain registers continue to pose challenges in particular for cross-lingual transfer. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|