Joint Learning of Pre-Trained and Random Units for Domain Adaptation in Part-of-Speech Tagging

Autor:	Meftah, Sara, Tamaazousti, Youssef, Semmar, Nasredine, Essafi, Hassane, Sadat, Fatiha
Rok vydání:	2019
Předmět:	Computer Science - Computation and Language Computer Science - Machine Learning Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Fine-tuning neural networks is widely used to transfer valuable knowledge from high-resource to low-resource domains. In a standard fine-tuning scheme, source and target problems are trained using the same architecture. Although capable of adapting to new domains, pre-trained units struggle with learning uncommon target-specific patterns. In this paper, we propose to augment the target-network with normalised, weighted and randomly initialised units that beget a better adaptation while maintaining the valuable source knowledge. Our experiments on POS tagging of social media texts (Tweets domain) demonstrate that our method achieves state-of-the-art performances on 3 commonly used datasets.
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1904.03595 Zobrazit plný text záznamu View this record from Arxiv