Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings
Autor: | Gonçalo Simões, Ryan McDonald, Bernd Bohnet, Daniel Andor, Emily Pitler, Joshua Maynez |
---|---|
Rok vydání: | 2018 |
Předmět: |
FOS: Computer and information sciences
Computer Science - Computation and Language Artificial neural network Computer science business.industry Context (language use) 02 engineering and technology Security token computer.software_genre Recurrent neural network Character (mathematics) 020204 information systems 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence business Representation (mathematics) Computation and Language (cs.CL) computer Word (computer architecture) Natural language processing |
Zdroj: | ACL (1) |
Popis: | The rise of neural networks, and particularly recurrent neural networks, has produced significant advances in part-of-speech tagging accuracy. One characteristic common among these models is the presence of rich initial word encodings. These encodings typically are composed of a recurrent character-based representation with learned and pre-trained word embeddings. However, these encodings do not consider a context wider than a single word and it is only through subsequent recurrent layers that word or sub-word information interacts. In this paper, we investigate models that use recurrent neural networks with sentence-level context for initial character and word-based representations. In particular we show that optimal results are obtained by integrating these context sensitive representations through synchronized training with a meta-model that learns to combine their states. We present results on part-of-speech and morphological tagging with state-of-the-art performance on a number of languages. |
Databáze: | OpenAIRE |
Externí odkaz: |