Scaling-up Memristor Monte Carlo with magnetic domain-wall physics
| Autor: | Dalgaty, Thomas, Yamada, Shogo, Molnos, Anca, Kawasaki, Eiji, Mesquida, Thomas, Rummens, François, Shibata, Tatsuo, Urakawa, Yukihiro, Terasaki, Yukio, Sasaki, Tomoyuki, Duranton, Marc |
|---|---|
| Rok vydání: | 2023 |
| Předmět: | |
| Druh dokumentu: | Working Paper |
| Popis: | By exploiting the intrinsic random nature of nanoscale devices, Memristor Monte Carlo (MMC) is a promising enabler of edge learning systems. However, due to multiple algorithmic and device-level limitations, existing demonstrations have been restricted to very small neural network models and datasets. We discuss these limitations, and describe how they can be overcome, by mapping the stochastic gradient Langevin dynamics (SGLD) algorithm onto the physics of magnetic domain-wall Memristors to scale-up MMC models by five orders of magnitude. We propose the push-pull pulse programming method that realises SGLD in-physics, and use it to train a domain-wall based ResNet18 on the CIFAR-10 dataset. On this task, we observe no performance degradation relative to a floating point model down to an update precision of between 6 and 7-bits, indicating we have made a step towards a large-scale edge learning system leveraging noisy analogue devices. Comment: Presented at the 1st workshop on Machine Learning with New Compute Paradigms (MLNCP) at NeurIPS 2023 (New Orleans, USA) |
| Databáze: | arXiv |
| Externí odkaz: |
|