Attending to Characters in Neural Sequence Labeling Models

Autor:	Rei, Marek, Crichton, Gamal K. O., Pyysalo, Sampo
Rok vydání:	2016
Předmět:	Computer Science - Computation and Language Computer Science - Learning Computer Science - Neural and Evolutionary Computing I.5.1 I.2.6 I.2.7
Druh dokumentu:	Working Paper
Popis:	Sequence labeling architectures use word embeddings for capturing similarity, but suffer when handling previously unseen or rare words. We investigate character-level extensions to such models and propose a novel architecture for combining alternative word representations. By using an attention mechanism, the model is able to dynamically decide how much information to use from a word- or character-level component. We evaluated different architectures on a range of sequence labeling datasets, and character-level extensions were found to improve performance on every benchmark. In addition, the proposed attention-based architecture delivered the best results even with a smaller number of trainable parameters. Comment: Proceedings of COLING 2016
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1611.04361 Zobrazit plný text záznamu View this record from Arxiv