Delving into Feature Maps: An Explanatory Analysis to Evaluate Weight Initialization
| Autor: | Prashant P. Bartakke, Meenal Narkhede, Mukul S. Sutaone |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Artificial neural network
Computer science business.industry Process (computing) Initialization 02 engineering and technology 010501 environmental sciences Machine learning computer.software_genre 01 natural sciences Convolutional neural network Standard deviation Maxima and minima Convergence (routing) 0202 electrical engineering electronic engineering information engineering Feature (machine learning) 020201 artificial intelligence & image processing Artificial intelligence business computer 0105 earth and related environmental sciences |
| Zdroj: | Advances in Intelligent Systems and Computing ISBN: 9783030736880 SoCPaR |
| DOI: | 10.1007/978-3-030-73689-7_29 |
| Popis: | Convolutional neural networks have delivered exceptional performance in various areas of computer vision. There has been growing research to develop deeper architectures with the availability of large datasets. Training such deep networks on large datasets is a tedious process as it involves optimizing a loss function by updating the parameters of the network. Weight initialization is a vital step before training neural networks as the correct choice of network weights ensures that the optimization converges to global minima in the least time. The weight initialization strategies in the literature can be categorized as (1) Initialization without pre-training, and (2) Initialization with pre-training. This paper presents a comparative analysis of the convergence performance of some widely used weight initialization techniques in these categories. This analysis is based on the diversity insights measured in terms of mean standard deviation captured from the feature maps. The experimentation has been carried out by training the AlexNet and VGG16 network on CIFAR-10 and CIFAR-100 datasets. The experimentation results demonstrate that the He initialization technique, which shows the best convergence performance among the others considered for the study, leads the training process such that the diversity of feature maps increases with epochs for both AlexNet and VGG16 network. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|