Convolutional Neural Network With Developmental Memory for Continual Learning
| Autor: | Gyeong-Moon Park, Sahng-Min Yoo, Jong-Hwan Kim |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Forgetting
Contextual image classification Computer Networks and Communications Computer science business.industry Knowledge engineering Feature extraction 02 engineering and technology Convolutional neural network Computer Science Applications Task (computing) Artificial Intelligence 0202 electrical engineering electronic engineering information engineering Task analysis 020201 artificial intelligence & image processing Artificial intelligence Transfer of learning business Software |
| Zdroj: | IEEE Transactions on Neural Networks and Learning Systems. 32:2691-2705 |
| ISSN: | 2162-2388 2162-237X |
| Popis: | Convolutional neural networks (CNNs) are one of the most successful deep neural networks. Indeed, most of the recent applications related to computer vision are based on CNNs. However, when learning new tasks in a sequential manner, CNNs face catastrophic forgetting: they forget a considerable amount of previously learned tasks while adapting to novel tasks. To overcome this main barrier to continual learning with CNNs, we introduce developmental memory (DM) into a CNN, continually generating submemory networks to learn important features of individual tasks. A novel training method, referred to here as guided learning (GL), guides the newly generated submemory to become an expert on the new task, eventually improving the performance of the overall network. At the same time, the existing submemories attempt to preserve the knowledge of old tasks. Experiments on image classification tasks show that compared with the state-of-the-art algorithms, the proposed CNN with DM not only improves the classification performance on the new image task but also leads to less forgetting of previous image tasks to facilitate continual learning. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|