Analysis of artificial neural network learning near temporary minima: A fuzzy logic approach

Autor:	Michael Margaliot, Ido Roth
Rok vydání:	2010
Předmět:	Artificial neural network Logic business.industry Fuzzy set Fuzzy control system Fuzzy logic Backpropagation Maxima and minima Artificial Intelligence Artificial intelligence Representation (mathematics) Gradient descent business Algorithm Mathematics
Zdroj:	Fuzzy Sets and Systems. 161:2569-2584
ISSN:	0165-0114
DOI:	10.1016/j.fss.2010.04.007
Popis:	Artificial neural networks (ANNs) are often trained using gradient descent algorithms (such as backpropagation). An important problem in the learning process is the slowdown incurred by temporary minima (TM). We analyze this problem for an ANN trained to solve the Exclusive Or problem. The network is transformed into the equivalent all permutations fuzzy rule-base (FARB), which provides a symbolic representation of the knowledge embedded in the network, after each learning step. We develop a mathematical model for the evolution of the fuzzy rule-base parameters during learning in the vicinity of TM. We show that the rule-base becomes singular and tends to remain singular in the vicinity of TM. The analysis of the fuzzy rule-base suggests a simple remedy for overcoming the slowdown in the learning process incurred by TM. This is based on slightly perturbing the desired output values in the training examples, so that they are no longer symmetric. Simulations demonstrate the effectiveness of this approach in reducing the time spent in the vicinity of TM.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_________::929747720f45b98f7226f7526be7ffc3 https://doi.org/10.1016/j.fss.2010.04.007 Zobrazit plný text záznamu Full Text from ScienceDirect