Accurate Supervised and Semi-Supervised Machine Reading for Long Documents
| Autor: | Llion Jones, Alexandre Lacoste, Izzeddin Gur, Daniel Hewlett |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
Sequence
Training set Computer science business.industry 05 social sciences Window (computing) Inference 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences Autoencoder 0502 economics and business State (computer science) Artificial intelligence 050207 economics business computer 0105 earth and related environmental sciences |
| Zdroj: | EMNLP |
| DOI: | 10.18653/v1/d17-1214 |
| Popis: | We introduce a hierarchical architecture for machine reading capable of extracting precise information from long documents. The model divides the document into small, overlapping windows and encodes all windows in parallel with an RNN. It then attends over these window encodings, reducing them to a single encoding, which is decoded into an answer using a sequence decoder. This hierarchical approach allows the model to scale to longer documents without increasing the number of sequential steps. In a supervised setting, our model achieves state of the art accuracy of 76.8 on the WikiReading dataset. We also evaluate the model in a semi-supervised setting by downsampling the WikiReading training set to create increasingly smaller amounts of supervision, while leaving the full unlabeled document corpus to train a sequence autoencoder on document windows. We evaluate models that can reuse autoencoder states and outputs without fine-tuning their weights, allowing for more efficient training and inference. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|