Training neural networks to encode symbols enables combinatorial generalization
| Autor: | Ivan Vankov, Jeffrey S. Bowers |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Symbolism Theoretical computer science Computer science Generalization Computer Science - Artificial Intelligence ENCODE Symbols 050105 experimental psychology General Biochemistry Genetics and Molecular Biology Combinatorial generalization 03 medical and health sciences 0302 clinical medicine Humans Learning 0501 psychology and cognitive sciences Computer Simulation Language Computer Science - Computation and Language Artificial neural network 05 social sciences Articles Artificial Intelligence (cs.AI) Core (graph theory) Cognitive Science Neural Networks Computer General Agricultural and Biological Sciences Computation and Language (cs.CL) Neural networks 030217 neurology & neurosurgery |
| Zdroj: | Philos Trans R Soc Lond B Biol Sci Sygma Explore Bristol Research Datacite UnpayWall ORCID Microsoft Academic Graph arXiv.org e-Print Archive European Union Open Data Portal PubMed Central Philosophical Transactions of the Royal Society B: Biological Sciences Vankov, I I & Bowers, J S 2019, ' Training neural networks to encode symbols enables combinatorial generalization ', Philosophical Transactions of the Royal Society B: Biological Sciences, vol. 375, no. 1791, 20190309 . https://doi.org/10.1098/rstb.2019.0309 |
| ISSN: | 1471-2970 0962-8436 |
| Popis: | Combinatorial generalization—the ability to understand and produce novel combinations of already familiar elements—is considered to be a core capacity of the human mind and a major challenge to neural network models. A significant body of research suggests that conventional neural networks cannot solve this problem unless they are endowed with mechanisms specifically engineered for the purpose of representing symbols. In this paper, we introduce a novel way of representing symbolic structures in connectionist terms—the vectors approach to representing symbols (VARS), which allows training standard neural architectures to encode symbolic knowledge explicitly at their output layers. In two simulations, we show that neural networks not only can learn to produce VARS representations, but in doing so they achieve combinatorial generalization in their symbolic and non-symbolic output. This adds to other recent work that has shown improved combinatorial generalization under some training conditions, and raises the question of whether specific mechanisms or training routines are needed to support symbolic processing. This article is part of the theme issue ‘Towards mechanistic models of meaning composition’. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|