Attention Inspired Network: Steep learning curve in an invariant pattern recognition model
| Autor: | Andreas Wichert, Luis Sa-Couto |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
0209 industrial biotechnology
Computer science Cognitive Neuroscience Feature extraction 02 engineering and technology Pattern Recognition Automated Machine Learning 020901 industrial engineering & automation Artificial Intelligence 0202 electrical engineering electronic engineering information engineering medicine Training set business.industry Deep learning Pattern recognition Invariant pattern recognition Visual cortex medicine.anatomical_structure Learning curve Test set 020201 artificial intelligence & image processing Neural Networks Computer Artificial intelligence business Classifier (UML) MNIST database |
| Zdroj: | Neural Networks. 114:38-46 |
| ISSN: | 0893-6080 |
| DOI: | 10.1016/j.neunet.2019.01.018 |
| Popis: | Hubel and Wiesel’s study about low areas of the visual cortex (VC) inspired deep models for invariant pattern recognition. In such models, simple and complex layers alternate local feature extraction with subsampling to add invariance to distortion or transformations. However, it was shown that to tolerate large changes between examples of the same category, the subsampling operation has to discard so much information that the model loses the capability to discriminate between categories. So, in practice, small changes are tolerated by these layers and, afterwards, a powerful classifier is introduced to do the rest. By incorporating insights from higher areas of the VC, we add to the already used retinotopic step an object-centered step which increases invariance capabilities without losing so much information. By doing so, we reduce the need for a powerful, data hungry classification layer and, thus, are able to introduce a simple classification mechanism which is based on selective attention. The resulting model is tested with an invariant pattern recognition task in the MNIST and ETL-1 datasets. We verify that the model is able to achieve better accuracies with less training examples. More specifically, on the MNIST test set, the model achieves a 100% accuracy when trained with little more than 10% of the training set. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect