Learning to Continually Learn Rapidly from Few and Noisy Data
| Autor: | Kuo, Nicholas I-Hsien, Harandi, Mehrtash, Fourrier, Nicolas, Walder, Christian, Ferraro, Gabriela, Suominen, Hanna |
|---|---|
| Rok vydání: | 2021 |
| Předmět: | |
| Druh dokumentu: | Working Paper |
| Popis: | Neural networks suffer from catastrophic forgetting and are unable to sequentially learn new tasks without guaranteed stationarity in data distribution. Continual learning could be achieved via replay -- by concurrently training externally stored old data while learning a new task. However, replay becomes less effective when each past task is allocated with less memory. To overcome this difficulty, we supplemented replay mechanics with meta-learning for rapid knowledge acquisition. By employing a meta-learner, which \textit{learns a learning rate per parameter per past task}, we found that base learners produced strong results when less memory was available. Additionally, our approach inherited several meta-learning advantages for continual learning: it demonstrated strong robustness to continually learn under the presence of noises and yielded base learners to higher accuracy in less updates. Comment: Accepted to the Meta-Learning and Co-Hosted Competition of AAAI 2021. See https://aaai.org/Conferences/AAAI-21/ws21workshops/ and see https://sites.google.com/chalearn.org/metalearning?pli=1#h.kt23ep5wlehv |
| Databáze: | arXiv |
| Externí odkaz: |
|