Post-training discriminative pruning for RBMs

Autor: Enrique Albornoz, Hugo Leonardo Rufiner, John Goddard Close, Máximo Sánchez-Gutiérrez
Rok vydání: 2017
Předmět:
0209 industrial biotechnology
Boltzmann machine
Word error rate
Computational intelligence
02 engineering and technology
Machine learning
computer.software_genre
Theoretical Computer Science
purl.org/becyt/ford/1 [https]
020901 industrial engineering & automation
Discriminative model
0202 electrical engineering
electronic engineering
information engineering
PRUNING
EMOTION CLASSIFICATION
DISCRIMINATIVE INFORMATION
PHONEME CLASSIFICATION
Mathematics
Network architecture
Artificial neural network
business.industry
Pattern recognition
purl.org/becyt/ford/1.2 [https]
Mutual information
Ciencias de la Computación
RESTRICTED BOLTZMANN MACHINES
Ciencias de la Computación e Información
020201 artificial intelligence & image processing
Geometry and Topology
Artificial intelligence
business
computer
Classifier (UML)
CIENCIAS NATURALES Y EXACTAS
Software
Zdroj: CONICET Digital (CONICET)
Consejo Nacional de Investigaciones Científicas y Técnicas
instacron:CONICET
ISSN: 1433-7479
1432-7643
DOI: 10.1007/s00500-017-2784-3
Popis: One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network. Fil: Sánchez Gutiérrez, Máximo. Universidad Autónoma Metropolitana; México Fil: Albornoz, Enrique Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina Fil: Rufiner, Hugo Leonardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina. Universidad Nacional de Entre Ríos; Argentina Fil: Close, John Goddard. Universidad Autónoma Metropolitana; México
Databáze: OpenAIRE
Externí odkaz: