| Autor: |
Basirat M; Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16/II, 8010 Graz, Austria., Geiger BC; Know-Center GmbH, Inffeldgasse 13, 8010 Graz, Austria., Roth PM; International AI Future Lab, Technical University of Munich (TUM), Willy-Messerschmitt-Straße 1, 85521 Taufkirchen, Germany. |
| Jazyk: |
angličtina |
| Zdroj: |
Entropy (Basel, Switzerland) [Entropy (Basel)] 2021 Jun 03; Vol. 23 (6). Date of Electronic Publication: 2021 Jun 03. |
| DOI: |
10.3390/e23060711 |
| Abstrakt: |
Information plane analysis, describing the mutual information between the input and a hidden layer and between a hidden layer and the target over time, has recently been proposed to analyze the training of neural networks. Since the activations of a hidden layer are typically continuous-valued, this mutual information cannot be computed analytically and must thus be estimated, resulting in apparently inconsistent or even contradicting results in the literature. The goal of this paper is to demonstrate how information plane analysis can still be a valuable tool for analyzing neural network training. To this end, we complement the prevailing binning estimator for mutual information with a geometric interpretation. With this geometric interpretation in mind, we evaluate the impact of regularization and interpret phenomena such as underfitting and overfitting. In addition, we investigate neural network learning in the presence of noisy data and noisy labels. |
| Databáze: |
MEDLINE |
| Externí odkaz: |
|
|
Nepřihlášeným uživatelům se plný text nezobrazuje |
K zobrazení výsledku je třeba se přihlásit.
|
