Thermodynamics of bidirectional associative memories
| Autor: | Adriano Barra, Giovanni Catania, Aurélien Decelle, Beatriz Seoane |
|---|---|
| Přispěvatelé: | Istituto Nazionale di Fisica Nucleare, Sezione di Lecce (INFN, Sezione di Lecce), Istituto Nazionale di Fisica Nucleare (INFN), Università del Salento [Lecce], Universidad Complutense de Madrid = Complutense University of Madrid [Madrid] (UCM), TAckling the Underspecified (TAU), Inria Saclay - Ile de France, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Interdisciplinaire des Sciences du Numérique (LISN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS), Laboratoire Interdisciplinaire des Sciences du Numérique (LISN), Institut National de Recherche en Informatique et en Automatique (Inria)-CentraleSupélec-Université Paris-Saclay-Centre National de la Recherche Scientifique (CNRS) |
| Rok vydání: | 2023 |
| Předmět: |
FOS: Computer and information sciences
Statistics and Probability Computer Science - Machine Learning Statistical Mechanics (cond-mat.stat-mech) Computer Science - Neural and Evolutionary Computing FOS: Physical sciences General Physics and Astronomy Statistical and Nonlinear Physics Disordered Systems and Neural Networks (cond-mat.dis-nn) Condensed Matter - Disordered Systems and Neural Networks Machine Learning (cs.LG) [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] Modeling and Simulation Neural and Evolutionary Computing (cs.NE) [PHYS.COND.CM-DS-NN]Physics [physics]/Condensed Matter [cond-mat]/Disordered Systems and Neural Networks [cond-mat.dis-nn] [PHYS.COND.CM-SM]Physics [physics]/Condensed Matter [cond-mat]/Statistical Mechanics [cond-mat.stat-mech] Condensed Matter - Statistical Mechanics Mathematical Physics |
| Zdroj: | Journal of Physics A: Mathematical and Theoretical. 56:205005 |
| ISSN: | 1751-8121 1751-8113 |
| Popis: | In this paper we investigate the equilibrium properties of bidirectional associative memories (BAMs). Introduced by Kosko in 1988 as a generalization of the Hopfield model to a bipartite structure, the simplest architecture is defined by two layers of neurons, with synaptic connections only between units of different layers: even without internal connections within each layer, information storage and retrieval are still possible through the reverberation of neural activities passing from one layer to another. We characterize the computational capabilities of a stochastic extension of this model in the thermodynamic limit, by applying rigorous techniques from statistical physics. A detailed picture of the phase diagram at the replica symmetric level is provided, both at finite temperature and in the noiseless regimes. Also for the latter, the critical load is further investigated up to one step of replica symmetry breaking. An analytical and numerical inspection of the transition curves (namely critical lines splitting the various modes of operation of the machine) is carried out as the control parameters - noise, load and asymmetry between the two layer sizes - are tuned. In particular, with a finite asymmetry between the two layers, it is shown how the BAM can store information more efficiently than the Hopfield model by requiring less parameters to encode a fixed number of patterns. Comparisons are made with numerical simulations of neural dynamics. Finally, a low-load analysis is carried out to explain the retrieval mechanism in the BAM by analogy with two interacting Hopfield models. A potential equivalence with two coupled Restricted Boltmzann Machines is also discussed. Comment: 25 pages, 11 figures |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|