融合节点与边特征的增强Graph Transformer

doi:10.6040/j.issn.1672-3961.0.2024.310

摘要/Abstract

摘要： 为解决邻域聚合Graph Transformer(neighborhood aggregation Graph Transformer, NAGphormer)缺乏对边特征的有效利用机制和图拉普拉斯矩阵的特征向量仅适用于无向图问题,本研究提出融合节点与边特征的增强Graph Transformer模型(enhanced Graph Transformer with node and edge feature fusion, EGT-NEF)。利用虚拟节点解决有向图中无法转换线图问题;引入线图映射矩阵使模型能够从邻域中学习边特征表示;通过图邻接矩阵的奇异值分解(singular value decomposition, SVD)来生成位置编码,将该模型扩展到有向图。结果表明,所提出的模型相较于基准模型在性能上有所提升。

关键词: 图神经网络, Graph Transformer, 虚拟节点, 边嵌入, SVD

Abstract: To address the problems that the neighborhood aggregation Graph Transformer(NAGphormer)lacked an effective mechanism for utilizing edge features and that the eigenvectors of the graph Laplacian matrix were only applicable to undirected graphs, this paper proposed a novel model, namely enhanced Graph Transformer with node and edge feature fusion(EGT-NEF). A dummy node was used to solve the problem that directed graphs could be converted into line graphs; the line graph mapping matrix was introduced to enable the model to learn edge features from neighborhoods; position encodings were generated through the singular value decomposition(SVD)of the graph adjacency matrix to extend the model to directed graphs. The experimental results showed that the proposed model achieved certain improvements in performance compared with the baseline.

Key words: graph neural network, Graph Transformer, dummy node, edge embedding, SVD

中图分类号:

TP183

黎俊亮,蒋沅,吴珑雪,刘宇. 融合节点与边特征的增强Graph Transformer[J]. 山东大学学报 (工学版), 2026, 56(3): 118-126.

LI Junliang, JIANG Yuan, WU Longxue, LIU Yu. Enhanced Graph Transformer with node and edge feature fusion[J]. Journal of Shandong University(Engineering Science), 2026, 56(3): 118-126.

参考文献

[1] KHEMANI B, PATIL S, KOTECHA K, et al. A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions[J]. Journal of Big Data, 2024, 11(1): 18.
[2] CHEN Y H, FRANCESCHI L, MINERVINI P, et al. ReFactor GNNs: revisiting factorisation-based models from a message-passing perspective[C] //Advances in Neural Information Processing Systems 35.New Orleans, USA: Neural Information Processing Systems Foun-dation, Inc.(NeurIPS), 2022: 16138-16150.
[3] EPPING B, RENÉ A, HELIAS M, et al. Graph neural networks do not always oversmooth[C] //Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2024: 48164-48188.
[4] ALON U, YAHAV E. On the bottleneck of graph neural networks and its practical implications [EB/OL].(2020-06-09)[2026-03-31]. https://arxiv.org/abs/2006. 05205
[5] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for Quantum chemistry[C] //Proceedings of the 34th International Conference on Machine Learning-Volume 70. Sydney, Australia: ACM, 2017: 1263-1272.
[6] 常新功,苏敏惠,周志刚. 基于进化集成的图神经网络解释方法[J]. 山东大学学报(工学版), 2024, 54(4): 1-12. CHANG Xingong, SU Minhui, ZHOU Zhigang. Explainer for GNN based on evolutionary ensemble learning algorithm[J]. Journal of Shandong University(Engineering Science), 2024, 54(4): 1-12.
[7] NIU Z, ZHONG G, YU H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62.
[8] LIN T Y, WANG Y X, LIU X Y, et al. A survey of transformers[J]. AI Open, 2022, 3: 111-132.
[9] SINHA A, YAMADA M, ZENG S Q, et al. Learning structured representations with hyperbolic embeddings[C] //Advances in Neural Information Processing Systems 37. Vancouver, BC, Canada: Neural Information Processing Systems Foundation, Inc.(NeurIPS), 2024: 91220-91259.
[10] SHEHZAD A, XIA F, ABID S, et al. Graph transformers: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2026(99): 1-20.
[11] MIN E, CHEN R, BIAN Y, et al. Transformer for graphs: An overview from architecture perspective [EB/OL].(2022-02-17)[2026-03-31]. https://arxiv.org/abs/2202. 08455
[12] 刘冬兰,刘新,刘家乐,等. 基于分解式Transformer的联邦长期时间序列预测算法[J]. 山东大学学报(工学版), 2024, 54(5): 101-110. LIU Donglan, LIU Xin, LIU Jiale, et al. Federated long-term time series forecasting algorithm based on decomposed Transformer[J]. Journal of Shandong University(Engineering Science), 2024, 54(5): 101-110.
[13] KAMBALE W V, KADURHA D K, EL BAHNASAWI M, et al. Transformers in time series forecasting: a brief transfer learning performance analysis[C] //2023 27th International Conference on Circuits, Systems, Communications and Computers(CSCC). Rhodes Island, Greece: IEEE, 2023: 212-217.
[14] CHEN J S, LIU C, GAO K Y, et al. NAGphormer: a tokenized graph transformer with neighborhood augmentation for node classification in large graphs[J]. IEEE Transactions on Big Data, 2025, 11(4): 2085-2098.
[15] ROSSI E, CHARPENTIER B, DI GIOVANNI F, et al. Edge directionality improves learning on heterophilic graphs[EB/OL].(2023-05-17)[2026-03-31]. https://arxiv.org/abs/2305.10498
[16] ZHOU Y C, HUO H T, HOU Z W, et al. Co-embedding of edges and nodes with deep graph convolutional neural networks[J]. Scientific Reports, 2023, 13: 16966.
[17] PARKS A D. Topological invariance under line graph transformations[J]. Symmetry, 2012, 4(2): 329-335.
[18] LIU X, CHENG J, SONG Y, et al. Boosting graph structure learning with dummy nodes[EB/OL].(2022-06-17)[2026-03-31]. https://arxiv.org/abs/2206. 08561
[19] HUSSAIN M S, HUSSAIN M S, ZAKI M J, et al. Global self-attention as a replacement for graph convolution[C] //Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington DC, USA: ACM, 2022: 655-665.
[20] KLICPERA J, BOJCHEVSKI A, GÜNNEMANN S. Predict then propagate: graph neural networks meet personalized pageRank[C] //International Conference on Learning Representations. Addis Ababa, Ethiopia: ICLR, 2019: 1-15.
[21] WU F, SOUZA A, ZHANG T, et al. Simplifying graph convolutional networks[C] //International Conference on Machine Learning. Long Beach, USA: PMLR, 2019: 6861-6871.
[22] SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data[J]. AI Magazine, 2008, 29(3): 93-106.
[23] HANG M Y, NEVILLE J, RIBEIRO B. A collective learning framework to boost GNN expressiveness for node classification[EB/OL].(2020-03-26)[2026-01-31]. https://arxiv.org/abs/2003.12169
[24] SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation [EB/OL].(2018-11-14)[2026-03-31]. https://arxiv.org/abs/1811. 05868
[25] FENG W Z, FENG W Z, ZHANG J, et al. Graph random neural networks for semi-supervised learning on graphs[C] //Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2020: 22092-22103.
[26] HAMILTON W L, HAMILTON W L, YING R, et al. Inductive representation learning on large graphs[C] //Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: ACM, 2017: 1025-1035.
[27] FENG W Z, FENG W Z, DONG Y X, et al. GRAND+: scalable graph random neural networks[C] //Proceedings of the ACM Web Conference 2022. Virtual Event, France:ACM, 2022: 3248-3258.
[28] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL].(2016-09-09)[2026-03-31]. https://arxiv.org/ abs/1609.02907
[29] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks [EB/OL].(2017-10-30)[2026-03-31]. https://arxiv.org/abs/1710.10903
[30] WANG Z M, CHEN J, CHEN H P. EGAT: edge-featured graph AttentionNetwork[C] //Artificial Neural Networks and Machine Learning-ICANN 2021. Cham, Switzerland: Springer, 2021: 253-264
[31] ZENG H, ZHOU H, SRIVASTAVA A, et al. GraphSAINT: graph sampling based inductive learning method [EB/OL].(2019-07-10)[2026-03-31]. https://arxiv.org/abs/1907.04931
[32] BOJCHEVSKIA, BOJCHEVSKI A, GASTEIGER J, et al. Scaling graph neural networks with approximate PageRank[C] //Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event, USA: ACM, 2020: 2464-2473.
[33] DWIVEDI V P, BRESSON X. A generalization of transformer networks to graphs [EB/OL].(2020-12-17)[2026-03-31]. https://arxiv.org/abs/2012.09699
[34] YING C, CAI T, LUO S, et al. Do transformers really perform badly for graph representation? [EB/OL].(2021-06-09)[2026-03-31]. https://10.48550/arXiv.2106.05234
[35] YING C, CAI T, LUO S, et al. Do transformers really perform badly for graph representation? [EB/OL].(2021-06-09)[2026-03-31]. https://10.48550/arXiv.2106.05234

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed