Class sparsity signature based Restricted Boltzmann Machine
| Autor: | Anush Sankaran, Richa Singh, Angshul Majumdar, Gaurav Goswami, Mayank Vatsa |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
Computer science
Boltzmann machine 02 engineering and technology Semi-supervised learning Overfitting Machine learning computer.software_genre Regularization (mathematics) Deep belief network symbols.namesake Discriminative model Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Restricted Boltzmann machine Training set business.industry Deep learning Supervised learning 020206 networking & telecommunications Pattern recognition ComputingMethodologies_PATTERNRECOGNITION Signal Processing Boltzmann constant symbols Unsupervised learning 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence business computer Software MNIST database |
| Zdroj: | Pattern Recognition. 61:674-685 |
| ISSN: | 0031-3203 |
| Popis: | Restricted Boltzmann Machines (RBMs) have been extensively utilized in machine learning as core units in constructing deep learning architectures such as Deep Boltzmann Machines (DBMs) and Deep Belief Networks (DBNs). However, they are prone to overfitting and several regularization techniques have been proposed to mitigate this effect. In this paper, we propose the semi-supervised class sparsity signature based RBM formulation by combining unsupervised generative training of the RBM with a supervised sparsity regularizer. The proposed approach, termed as cssRBM, enforces sparsity at the class level to ensure that coherent and discriminative representations are learnt during training. Combining unsupervised learning with supervised learning allows the model to utilize external training data to learn better generative features while the supervised learning enables fine-tuning for discrimination using the learned features. We construct both DBMs and DBNs with cssRBM units and evaluate the performance on multiple publicly available benchmark datasets. Experiments on the MNIST and CIFAR-10 databases demonstrate that the proposed approaches are comparable with state-of-the-art deep learning architectures in the literature. We also evaluate the performance on one of the most challenging face databases, i.e., the Point and Shoot Challenge dataset. The results show that the proposed approaches improve state-of-the-art results by 15% on the PaSC database. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect