Variants of DropConnect in Learning vector quantization networks for evaluation of classification stability
| Autor: | Jensun Ravichandran, Marika Kaden, Sascha Saralajew, Thomas Villmann |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
0209 industrial biotechnology
Learning vector quantization Artificial neural network Computer science business.industry Cognitive Neuroscience Quantization (signal processing) Pattern recognition 02 engineering and technology Overfitting Computer Science Applications Support vector machine 020901 industrial engineering & automation Artificial Intelligence Robustness (computer science) Multilayer perceptron 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Artificial intelligence business MNIST database Interpretability |
| Zdroj: | Neurocomputing. 403:121-132 |
| ISSN: | 0925-2312 |
| DOI: | 10.1016/j.neucom.2019.12.131 |
| Popis: | Dropout and DropConnect are useful methods to prevent multilayer neural networks from overfitting. In addition, it turns out that these tools can also be used to estimate the stability of networks regarding disturbances. Prototype based networks gain more and more attraction in current research because of their inherent interpretability and robust behavior. Popular prototype-based classifiers are support vector machines and the heuristically motivated Learning Vector Quantizer (LVQ). The Generalized Matrix LVQ (GMLVQ) is an extension of LVQ which can be interpreted as a special multilayer network containing a projection and a prototype layer. First in this paper, we extend the linear projection layer of GMLVQ to a non-linear mapping by employing different non-linear activations functions. Second, we compare the classification decision stabilities of the linear and the non-linear GMLVQ regarding DropConnect while taking the neural network perspective. Thus we can adopt DropConnect ideas known from multilayer perceptron learning to investigate stability and robustness of GMLVQ. To this end, the evaluation of the stability is done in terms of a information theoretic stability measure based on the Shannon-Entropy. We demonstrate the approach for three real world data sets from Raman spectroscopy, multi-spectral remote sensing and the well-known MNIST data set. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect