Class-Incremental Learning based on Label Generation

Autor:	Shao, Yijia, Guo, Yiduo, Zhao, Dongyan, Liu, Bing
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Machine Learning Computer Science - Neural and Evolutionary Computing
Druh dokumentu:	Working Paper
Popis:	Despite the great success of pre-trained language models, it is still a challenge to use these models for continual learning, especially for the class-incremental learning (CIL) setting due to catastrophic forgetting (CF). This paper reports our finding that if we formulate CIL as a continual label generation problem, CF is drastically reduced and the generalizable representations of pre-trained models can be better retained. We thus propose a new CIL method (VAG) that also leverages the sparsity of vocabulary to focus the generation and creates pseudo-replay samples by using label semantics. Experimental results show that VAG outperforms baselines by a large margin. Comment: 12 pages, ACL 2023 Main Conference
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2306.12619 Zobrazit plný text záznamu View this record from Arxiv