Optimization of deep network models through fine tuning
| Autor: | M. Arif Wani, Saduf Afzal |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Restricted Boltzmann machine
General Computer Science Artificial neural network business.industry Computer science Deep learning 02 engineering and technology 010501 environmental sciences computer.software_genre 01 natural sciences Backpropagation 0202 electrical engineering electronic engineering information engineering Unsupervised learning 020201 artificial intelligence & image processing Artificial intelligence Data mining business computer Dropout (neural networks) MNIST database 0105 earth and related environmental sciences Network model |
| Zdroj: | International Journal of Intelligent Computing and Cybernetics. 11:386-403 |
| ISSN: | 1756-378X |
| DOI: | 10.1108/ijicc-06-2017-0070 |
| Popis: | Purpose Many strategies have been put forward for training deep network models, however, stacking of several layers of non-linearities typically results in poor propagation of gradients and activations. The purpose of this paper is to explore the use of two steps strategy where initial deep learning model is obtained first by unsupervised learning and then optimizing the initial deep learning model by fine tuning. A number of fine tuning algorithms are explored in this work for optimizing deep learning models. This includes proposing a new algorithm where Backpropagation with adaptive gain algorithm is integrated with Dropout technique and the authors evaluate its performance in the fine tuning of the pretrained deep network. Design/methodology/approach The parameters of deep neural networks are first learnt using greedy layer-wise unsupervised pretraining. The proposed technique is then used to perform supervised fine tuning of the deep neural network model. Extensive experimental study is performed to evaluate the performance of the proposed fine tuning technique on three benchmark data sets: USPS, Gisette and MNIST. The authors have tested the approach on varying size data sets which include randomly chosen training samples of size 20, 50, 70 and 100 percent from the original data set. Findings Through extensive experimental study, it is concluded that the two steps strategy and the proposed fine tuning technique significantly yield promising results in optimization of deep network models. Originality/value This paper proposes employing several algorithms for fine tuning of deep network model. A new approach that integrates adaptive gain Backpropagation (BP) algorithm with Dropout technique is proposed for fine tuning of deep networks. Evaluation and comparison of various algorithms proposed for fine tuning on three benchmark data sets is presented in the paper. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|