Syntactic Data Augmentation Increases Robustness to Inference Heuristics
| Autor: | Emily Pitler, Tal Linzen, Junghyun Min, Dipanjan Das, R. Thomas McCoy |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Computation and Language business.industry Computer science Inference 02 engineering and technology computer.software_genre Syntax Robustness (computer science) 020204 information systems Test set 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Syntactic structure Artificial intelligence Heuristics business Computation and Language (cs.CL) computer Natural language processing Word order |
| Zdroj: | ACL |
| DOI: | 10.18653/v1/2020.acl-main.212 |
| Popis: | Pretrained neural models such as BERT, when fine-tuned to perform natural language inference (NLI), often show high accuracy on standard datasets, but display a surprising lack of sensitivity to word order on controlled challenge sets. We hypothesize that this issue is not primarily caused by the pretrained model's limitations, but rather by the paucity of crowdsourced NLI examples that might convey the importance of syntactic structure at the fine-tuning stage. We explore several methods to augment standard training sets with syntactically informative examples, generated by applying syntactic transformations to sentences from the MNLI corpus. The best-performing augmentation method, subject/object inversion, improved BERT's accuracy on controlled examples that diagnose sensitivity to word order from 0.28 to 0.73, without affecting performance on the MNLI test set. This improvement generalized beyond the particular construction used for data augmentation, suggesting that augmentation causes BERT to recruit abstract syntactic representations. ACL 2020 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|