Information theoretical performance measure for associative memories and its application to neural networks
| Autor: | Oliver Kerschhaggl, Friedrich Wagner, Mathias Schlüter |
|---|---|
| Rok vydání: | 1999 |
| Předmět: |
Models
Statistical Theoretical computer science Artificial neural network Numerical analysis Information Theory Complex system Association Learning Observable Information loss Content-addressable memory Hopfield network Artificial Intelligence Memory Applied mathematics Neural Networks Computer Associative property Mathematics |
| Zdroj: | Physical Review E. 60:2141-2147 |
| ISSN: | 1095-3787 1063-651X |
| DOI: | 10.1103/physreve.60.2141 |
| Popis: | We present a general performance measure (information loss) for associative memories based on information theoretical concepts. This performance measure can be estimated, provided that mean values of observables have been determined for the associative memory. Then the estimation guarantees a minimal association quality. The formalism allows the application of the performance measure to complex systems where the relation between input and output of the associative memory is not explicitly known. Here we apply our formalism to the Hopfield model and estimate the storage capacity ${\ensuremath{\alpha}}_{c}$ from the numerically determined information loss. In contrast to other numerical methods the whole overlap distribution is taken into account. Our numerical value ${\ensuremath{\alpha}}_{c}=0.1379(4)$ for the storage capacity in the Hopfield model is below numerical values obtained previously. This indicates that the consideration of small remnant overlaps lowers the storage capacity of the Hopfield model. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|