Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations
| Autor: | Zuyi Bao, Huang Rui, Kenny Q. Zhu, Chen Li |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Matching (statistics) Computer Science - Computation and Language Computer science Low resource business.industry 02 engineering and technology computer.software_genre Sequence labeling 03 medical and health sciences 0302 clinical medicine 030221 ophthalmology & optometry 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence Language model business computer Computation and Language (cs.CL) Natural language processing |
| Zdroj: | EMNLP/IJCNLP (1) |
| DOI: | 10.48550/arxiv.1910.10893 |
| Popis: | Previous work on cross-lingual sequence labeling tasks either requires parallel data or bridges the two languages through word-byword matching. Such requirements and assumptions are infeasible for most languages, especially for languages with large linguistic distances, e.g., English and Chinese. In this work, we propose a Multilingual Language Model with deep semantic Alignment (MLMA) to generate language-independent representations for cross-lingual sequence labeling. Our methods require only monolingual corpora with no bilingual resources at all and take advantage of deep contextualized representations. Experimental results show that our approach achieves new state-of-the-art NER and POS performance across European languages, and is also effective on distant language pairs such as English and Chinese. Comment: Accepted at EMNLP 2019 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|