L1/2 regularization learning for smoothing interval neural networks: Algorithms and convergence analysis
| Autor: | Dakun Yang, Yan Liu |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
0209 industrial biotechnology
Artificial neural network Gradient learning Cognitive Neuroscience 02 engineering and technology Continuous derivative Regularization (mathematics) Computer Science Applications Interval arithmetic 020901 industrial engineering & automation Artificial Intelligence 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Algorithm Smoothing Mathematics |
| Zdroj: | Neurocomputing. 272:122-129 |
| ISSN: | 0925-2312 |
| DOI: | 10.1016/j.neucom.2017.06.061 |
| Popis: | Interval neural networks can easily address uncertain information, since they are capable of handling various kinds of uncertainties inherently which are represented by interval. L q (0 q L 1 regularization for better solution of sparsity problems, among which L 1/2 is of extreme importance and can be taken as a representative. However, weights oscillation might occur during learning process due to discontinuous derivative for L 1/2 regularization. In this paper, a novel batch gradient algorithm with smoothing L 1/2 regularization is proposed to prevent the weights oscillation for a smoothing interval neural network (SINN), which is the modified interval neural network. Here, by smoothing we mean that, in a neighborhood of the origin, we replace the absolute values of the weights by a smooth function for continuous derivative. Compared with conventional gradient learning algorithm with L 1/2 regularization, this approach can obtain sparser weights and simpler structure, and improve the learning efficiency. Then we present a sufficient condition for convergence of SINN. Finally, simulation results illustrate the convergence of the main results. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect