Exploring the efficiency of the combined application of connection pruning and source data preprocessing when training a multilayer perceptron
| Autor: | Oleg Galchonkov, Alexander Nevrev, Maria Glava, Mykola Babych |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Source data
neural network Computer science 020209 energy pruning 0211 other engineering and technologies Energy Engineering and Power Technology 02 engineering and technology Industrial and Manufacturing Engineering Management of Technology and Innovation lcsh:Technology (General) 021105 building & construction 0202 electrical engineering electronic engineering information engineering Redundancy (engineering) lcsh:Industry multilayer perceptron Electrical and Electronic Engineering Cumulative effect Artificial neural network business.industry Applied Mathematics Mechanical Engineering Pattern recognition Computer Science Applications regularization learning curve Control and Systems Engineering Learning curve weight coefficients Multilayer perceptron lcsh:T1-995 lcsh:HD2321-4730.9 Artificial intelligence business |
| Zdroj: | Eastern-European Journal of Enterprise Technologies, Vol 2, Iss 9 (104), Pp 6-13 (2020) |
| ISSN: | 1729-4061 1729-3774 |
| DOI: | 10.15587/1729-4061.2020.200819 |
| Popis: | A conventional scheme to operate neural networks until recently has been assigning the architecture of a neural network and its subsequent training. However, the latest research in this field has revealed that the neural networks that had been set and configured in this way exhibited considerable redundancy. Therefore, the additional operation was to eliminate this redundancy by pruning the connections in the architecture of a neural network. Among the many approaches to eliminating redundancy, the most promising one is the combined application of several methods when their cumulative effect exceeds the sum of effects from employing each of them separately. We have performed an experimental study into the effectiveness of the combined application of iterative pruning and pre-processing (pre-distortions) of input data for the task of recognizing handwritten digits with the help of a multilayer perceptron. It has been shown that the use of input data pre-processing regularizes the procedure of training a neural network, thereby preventing its retraining. The combined application of the iterative pruning and pre-processing of input data has made it possible to obtain a smaller error in the recognition of handwritten digits, 1.22%, compared to when using the thinning only (the error decreased from 1.89% to 1.81%) and when employing the predistortions only (the error decreased from 1.89% to 1.52%). In addition, the regularization involving pre-distortions makes it possible to receive a monotonously increasing number of disconnected connections while maintaining the error at 1.45%. The resulting learning curves for the same task but corresponding to the onset of training under different initial conditions acquire different values both in the learning process and at the end of the training. This shows the multi-extreme character of the quality function – the accuracy of recognition. The practical implication of the study is our proposal to run the multiple training of a neural network in order to choose the best result |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|