Adaptive Transfer Learning on Graph Neural Networks
| Autor: | Xueting Han, Jing Bai, Zhenhuan Huang, Bang An |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Computer Science - Machine Learning Computer science business.industry Multi-task learning Machine learning computer.software_genre Machine Learning (cs.LG) Weighting Consistency (database systems) Task (computing) Leverage (statistics) Artificial intelligence Adaptation (computer science) Representation (mathematics) Transfer of learning business computer |
| Zdroj: | KDD |
| Popis: | Graph neural networks (GNNs) is widely used to learn a powerful representation of graph-structured data. Recent work demonstrates that transferring knowledge from self-supervised tasks to downstream tasks could further improve graph representation. However, there is an inherent gap between self-supervised tasks and downstream tasks in terms of optimization objective and training data. Conventional pre-training methods may be not effective enough on knowledge transfer since they do not make any adaptation for downstream tasks. To solve such problems, we propose a new transfer learning paradigm on GNNs which could effectively leverage self-supervised tasks as auxiliary tasks to help the target task. Our methods would adaptively select and combine different auxiliary tasks with the target task in the fine-tuning stage. We design an adaptive auxiliary loss weighting model to learn the weights of auxiliary tasks by quantifying the consistency between auxiliary tasks and the target task. In addition, we learn the weighting model through meta-learning. Our methods can be applied to various transfer learning approaches, it performs well not only in multi-task learning but also in pre-training and fine-tuning. Comprehensive experiments on multiple downstream tasks demonstrate that the proposed methods can effectively combine auxiliary tasks with the target task and significantly improve the performance compared to state-of-the-art methods. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|