CP-Prompt: Composition-Based Cross-modal Prompting for Domain-Incremental Continual Learning

Autor:	Feng, Yu, Tian, Zhen, Zhu, Yifan, Han, Zongfu, Luo, Haoran, Zhang, Guangwei, Song, Meina
Rok vydání:	2024
Předmět:	Computer Science - Computation and Language Computer Science - Artificial Intelligence Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	The key challenge of cross-modal domain-incremental learning (DIL) is to enable the learning model to continuously learn from novel data with different feature distributions under the same task without forgetting old ones. However, existing top-performing methods still cause high forgetting rates, by lacking intra-domain knowledge extraction and inter-domain common prompting strategy. In this paper, we propose a simple yet effective framework, CP-Prompt, by training limited parameters to instruct a pre-trained model to learn new domains and avoid forgetting existing feature distributions. CP-Prompt captures intra-domain knowledge by compositionally inserting personalized prompts on multi-head self-attention layers and then learns the inter-domain knowledge with a common prompting strategy. CP-Prompt shows superiority compared with state-of-the-art baselines among three widely evaluated DIL tasks. The source code is available at https://github.com/dannis97500/CP_Prompt. Comment: Accepted by ACM MM 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.21043 Zobrazit plný text záznamu View this record from Arxiv