Convolution by Evolution
| Autor: | Dylan Banarse, Chrisantha Fernando, Malcolm Reynolds, Max Jaderberg, David Pfau, Marc Lanctot, Daan Wierstra, Frederic Besse |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
FOS: Computer and information sciences
0301 basic medicine Network architecture Computer science Generalization Computer Vision and Pattern Recognition (cs.CV) Computer Science - Computer Vision and Pattern Recognition Computer Science - Neural and Evolutionary Computing 02 engineering and technology Regularization (mathematics) Machine Learning (cs.LG) Convolution Image (mathematics) Computer Science - Learning 03 medical and health sciences 030104 developmental biology 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Neural and Evolutionary Computing (cs.NE) Differentiable function Compositional pattern-producing network Algorithm MNIST database |
| Zdroj: | GECCO |
| DOI: | 10.1145/2908812.2908890 |
| Popis: | In this work we introduce a differentiable version of the Compositional Pattern Producing Network, called the DPPN. Unlike a standard CPPN, the topology of a DPPN is evolved but the weights are learned. A Lamarckian algorithm, that combines evolution and learning, produces DPPNs to reconstruct an image. Our main result is that DPPNs can be evolved/trained to compress the weights of a denoising autoencoder from 157684 to roughly 200 parameters, while achieving a reconstruction accuracy comparable to a fully connected network with more than two orders of magnitude more parameters. The regularization ability of the DPPN allows it to rediscover (approximate) convolutional network architectures embedded within a fully connected architecture. Such convolutional architectures are the current state of the art for many computer vision applications, so it is satisfying that DPPNs are capable of discovering this structure rather than having to build it in by design. DPPNs exhibit better generalization when tested on the Omniglot dataset after being trained on MNIST, than directly encoded fully connected autoencoders. DPPNs are therefore a new framework for integrating learning and evolution. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|