Robust High-dimensional Memory-augmented Neural Networks
| Autor: | Manuel Schmuck, Manuel Le Gallo, Giovanni Cherubini, Abu Sebastian, Abbas Rahimi, Geethan Karunaratne, Luca Benini |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Matching (graph theory) Computer science Science Computation General Physics and Astronomy Computer Science - Emerging Technologies 02 engineering and technology General Biochemistry Genetics and Molecular Biology Bottleneck Article Machine Learning (cs.LG) symbols.namesake Software 0202 electrical engineering electronic engineering information engineering Explicit memory Neural and Evolutionary Computing (cs.NE) Multidisciplinary Artificial neural network Contextual image classification business.industry Computer Science - Neural and Evolutionary Computing General Chemistry 021001 nanoscience & nanotechnology Electrical and electronic engineering 020202 computer hardware & architecture Emerging Technologies (cs.ET) Computer engineering symbols 0210 nano-technology business Von Neumann architecture |
| Zdroj: | Nature Communications Nature Communications, 12 (1) Nature Communications, Vol 12, Iss 1, Pp 1-12 (2021) |
| ISSN: | 2041-1723 |
| Popis: | Traditional neural networks require enormous amounts of data to build their complex mappings during a slow training procedure that hinders their abilities for relearning and adapting to new data. Memory-augmented neural networks enhance neural networks with an explicit memory to overcome these issues. Access to this explicit memory, however, occurs via soft read and write operations involving every individual memory entry, resulting in a bottleneck when implemented using the conventional von Neumann computer architecture. To overcome this bottleneck, we propose a robust architecture that employs a computational memory unit as the explicit memory performing analog in-memory computation on high-dimensional (HD) vectors, while closely matching 32-bit software-equivalent accuracy. This is achieved by a content-based attention mechanism that represents unrelated items in the computational memory with uncorrelated HD vectors, whose real-valued components can be readily approximated by binary, or bipolar components. Experimental results demonstrate the efficacy of our approach on few-shot image classification tasks on the Omniglot dataset using more than 256,000 phase-change memory devices. Our approach effectively merges the richness of deep neural network representations with HD computing that paves the way for robust vector-symbolic manipulations applicable in reasoning, fusion, and compression. Nature Communications, 12 (1) ISSN:2041-1723 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|