Modular representation of layered neural networks

Autor: Kunio Kashino, Kaoru Hiramatsu, Chihiro Watanabe
Rok vydání: 2018
Předmět:
FOS: Computer and information sciences
Computer science
Cognitive Neuroscience
Machine Learning (stat.ML)
02 engineering and technology
computer.software_genre
Machine Learning (cs.LG)
Machine Learning
Probabilistic neural network
Statistics - Machine Learning
Artificial Intelligence
Image Processing
Computer-Assisted
0202 electrical engineering
electronic engineering
information engineering
Cluster Analysis
Stochastic neural network
Modularity (networks)
Artificial neural network
business.industry
Time delay neural network
Deep learning
Computational Biology
020206 networking & telecommunications
Computer Science - Learning
Nonlinear system
Recurrent neural network
Nonlinear Dynamics
020201 artificial intelligence & image processing
Neural Networks
Computer
Data mining
Artificial intelligence
business
computer
Algorithms
Nervous system network models
Zdroj: Neural Networks. 97:62-73
ISSN: 0893-6080
DOI: 10.1016/j.neunet.2017.09.017
Popis: Layered neural networks have greatly improved the performance of various applications including image processing, speech recognition, natural language processing, and bioinformatics. However, it is still difficult to discover or interpret knowledge from the inference provided by a layered neural network, since its internal representation has many nonlinear and complex parameters embedded in hierarchical layers. Therefore, it becomes important to establish a new methodology by which layered neural networks can be understood. In this paper, we propose a new method for extracting a global and simplified structure from a layered neural network. Based on network analysis, the proposed method detects communities or clusters of units with similar connection patterns. We show its effectiveness by applying it to three use cases. (1) Network decomposition: it can decompose a trained neural network into multiple small independent networks thus dividing the problem and reducing the computation time. (2) Training assessment: the appropriateness of a trained result with a given hyperparameter or randomly chosen initial parameters can be evaluated by using a modularity index. And (3) data analysis: in practical data it reveals the community structure in the input, hidden, and output layers, which serves as a clue for discovering knowledge from a trained neural network.
Databáze: OpenAIRE
Externí odkaz: