Graph Propagation Transformer for Graph Representation Learning

Autor:	Chen, Zhe, Tan, Hao, Wang, Tao, Shen, Tianrun, Lu, Tong, Peng, Qiuying, Cheng, Cheng, Qi, Yue
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence
Druh dokumentu:	Working Paper
Popis:	This paper presents a novel transformer architecture for graph representation learning. The core insight of our method is to fully consider the information propagation among nodes and edges in a graph when building the attention module in the transformer blocks. Specifically, we propose a new attention mechanism called Graph Propagation Attention (GPA). It explicitly passes the information among nodes and edges in three ways, i.e. node-to-node, node-to-edge, and edge-to-node, which is essential for learning graph-structured data. On this basis, we design an effective transformer architecture named Graph Propagation Transformer (GPTrans) to further help learn graph data. We verify the performance of GPTrans in a wide range of graph learning experiments on several benchmark datasets. These results show that our method outperforms many state-of-the-art transformer-based graph models with better performance. The code will be released at https://github.com/czczup/GPTrans. Comment: Accepted to IJCAI 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2305.11424 Zobrazit plný text záznamu View this record from Arxiv