WeightScale: Interpreting Weight Change in Neural Networks

Autor:	Agrawal, Ayush Manish, Tendle, Atharva, Sikka, Harshvardhan, Singh, Sahib
Rok vydání:	2021
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	Interpreting the learning dynamics of neural networks can provide useful insights into how networks learn and the development of better training and design approaches. We present an approach to interpret learning in neural networks by measuring relative weight change on a per layer basis and dynamically aggregating emerging trends through combination of dimensionality reduction and clustering which allows us to scale to very deep networks. We use this approach to investigate learning in the context of vision tasks across a variety of state-of-the-art networks and provide insights into the learning behavior of these networks, including how task complexity affects layer-wise learning in deeper layers of networks. Comment: Intelligent Computing, 2021. arXiv admin note: text overlap with arXiv:2011.06735
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2107.07005 Zobrazit plný text záznamu View this record from Arxiv