LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models

Autor:	Ontanon, Santiago, Ainslie, Joshua, Cvicek, Vaclav, Fisher, Zachary
Rok vydání:	2022
Předmět:	Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Machine learning models such as Transformers or LSTMs struggle with tasks that are compositional in nature such as those involving reasoning/inference. Although many datasets exist to evaluate compositional generalization, when it comes to evaluating inference abilities, options are more limited. This paper presents LogicInference, a new dataset to evaluate the ability of models to perform logical inference. The dataset focuses on inference using propositional logic and a small subset of first-order logic, represented both in semi-formal logical notation, as well as in natural language. We also report initial results using a collection of machine learning models to establish an initial baseline in this dataset. Comment: Accepted at ICLR 2022 OSC workshop (v3 contains updated results after fixing a problem in dataset generation)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2203.15099 Zobrazit plný text záznamu View this record from Arxiv