Journal of Shandong University(Engineering Science) ›› 2026, Vol. 56 ›› Issue (3): 118-126.doi: 10.6040/j.issn.1672-3961.0.2024.310

• Machine Learning & Data Mining • Previous Articles     Next Articles

Enhanced Graph Transformer with node and edge feature fusion

LI Junliang, JIANG Yuan*, WU Longxue, LIU Yu   

  1. LI Junliang, JIANG Yuan*, WU Longxue, LIU Yu(School of Information Engineering, Nanchang Hangkong University, Nanchang 330063, Jiangxi, China
  • Published:2026-06-09

PDF (PC)

2

Abstract: To address the problems that the neighborhood aggregation Graph Transformer(NAGphormer)lacked an effective mechanism for utilizing edge features and that the eigenvectors of the graph Laplacian matrix were only applicable to undirected graphs, this paper proposed a novel model, namely enhanced Graph Transformer with node and edge feature fusion(EGT-NEF). A dummy node was used to solve the problem that directed graphs could be converted into line graphs; the line graph mapping matrix was introduced to enable the model to learn edge features from neighborhoods; position encodings were generated through the singular value decomposition(SVD)of the graph adjacency matrix to extend the model to directed graphs. The experimental results showed that the proposed model achieved certain improvements in performance compared with the baseline.

Key words: graph neural network, Graph Transformer, dummy node, edge embedding, SVD

CLC Number: 

  • TP183
[1] KHEMANI B, PATIL S, KOTECHA K, et al. A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions[J]. Journal of Big Data, 2024, 11(1): 18.
[2] CHEN Y H, FRANCESCHI L, MINERVINI P, et al. ReFactor GNNs: revisiting factorisation-based models from a message-passing perspective[C] //Advances in Neural Information Processing Systems 35.New Orleans, USA: Neural Information Processing Systems Foun-dation, Inc.(NeurIPS), 2022: 16138-16150.
[3] EPPING B, RENÉ A, HELIAS M, et al. Graph neural networks do not always oversmooth[C] //Proceedings of the 38th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2024: 48164-48188.
[4] ALON U, YAHAV E. On the bottleneck of graph neural networks and its practical implications [EB/OL].(2020-06-09)[2026-03-31]. https://arxiv.org/abs/2006. 05205
[5] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for Quantum chemistry[C] //Proceedings of the 34th International Conference on Machine Learning-Volume 70. Sydney, Australia: ACM, 2017: 1263-1272.
[6] 常新功,苏敏惠,周志刚. 基于进化集成的图神经网络解释方法[J]. 山东大学学报(工学版), 2024, 54(4): 1-12. CHANG Xingong, SU Minhui, ZHOU Zhigang. Explainer for GNN based on evolutionary ensemble learning algorithm[J]. Journal of Shandong University(Engineering Science), 2024, 54(4): 1-12.
[7] NIU Z, ZHONG G, YU H. A review on the attention mechanism of deep learning[J]. Neurocomputing, 2021, 452: 48-62.
[8] LIN T Y, WANG Y X, LIU X Y, et al. A survey of transformers[J]. AI Open, 2022, 3: 111-132.
[9] SINHA A, YAMADA M, ZENG S Q, et al. Learning structured representations with hyperbolic embeddings[C] //Advances in Neural Information Processing Systems 37. Vancouver, BC, Canada: Neural Information Processing Systems Foundation, Inc.(NeurIPS), 2024: 91220-91259.
[10] SHEHZAD A, XIA F, ABID S, et al. Graph transformers: a survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2026(99): 1-20.
[11] MIN E, CHEN R, BIAN Y, et al. Transformer for graphs: An overview from architecture perspective [EB/OL].(2022-02-17)[2026-03-31]. https://arxiv.org/abs/2202. 08455
[12] 刘冬兰,刘新,刘家乐,等. 基于分解式Transformer的联邦长期时间序列预测算法[J]. 山东大学学报(工学版), 2024, 54(5): 101-110. LIU Donglan, LIU Xin, LIU Jiale, et al. Federated long-term time series forecasting algorithm based on decomposed Transformer[J]. Journal of Shandong University(Engineering Science), 2024, 54(5): 101-110.
[13] KAMBALE W V, KADURHA D K, EL BAHNASAWI M, et al. Transformers in time series forecasting: a brief transfer learning performance analysis[C] //2023 27th International Conference on Circuits, Systems, Communications and Computers(CSCC). Rhodes Island, Greece: IEEE, 2023: 212-217.
[14] CHEN J S, LIU C, GAO K Y, et al. NAGphormer: a tokenized graph transformer with neighborhood augmentation for node classification in large graphs[J]. IEEE Transactions on Big Data, 2025, 11(4): 2085-2098.
[15] ROSSI E, CHARPENTIER B, DI GIOVANNI F, et al. Edge directionality improves learning on heterophilic graphs[EB/OL].(2023-05-17)[2026-03-31]. https://arxiv.org/abs/2305.10498
[16] ZHOU Y C, HUO H T, HOU Z W, et al. Co-embedding of edges and nodes with deep graph convolutional neural networks[J]. Scientific Reports, 2023, 13: 16966.
[17] PARKS A D. Topological invariance under line graph transformations[J]. Symmetry, 2012, 4(2): 329-335.
[18] LIU X, CHENG J, SONG Y, et al. Boosting graph structure learning with dummy nodes[EB/OL].(2022-06-17)[2026-03-31]. https://arxiv.org/abs/2206. 08561
[19] HUSSAIN M S, HUSSAIN M S, ZAKI M J, et al. Global self-attention as a replacement for graph convolution[C] //Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Washington DC, USA: ACM, 2022: 655-665.
[20] KLICPERA J, BOJCHEVSKI A, GÜNNEMANN S. Predict then propagate: graph neural networks meet personalized pageRank[C] //International Conference on Learning Representations. Addis Ababa, Ethiopia: ICLR, 2019: 1-15.
[21] WU F, SOUZA A, ZHANG T, et al. Simplifying graph convolutional networks[C] //International Conference on Machine Learning. Long Beach, USA: PMLR, 2019: 6861-6871.
[22] SEN P, NAMATA G, BILGIC M, et al. Collective classification in network data[J]. AI Magazine, 2008, 29(3): 93-106.
[23] HANG M Y, NEVILLE J, RIBEIRO B. A collective learning framework to boost GNN expressiveness for node classification[EB/OL].(2020-03-26)[2026-01-31]. https://arxiv.org/abs/2003.12169
[24] SHCHUR O, MUMME M, BOJCHEVSKI A, et al. Pitfalls of graph neural network evaluation [EB/OL].(2018-11-14)[2026-03-31]. https://arxiv.org/abs/1811. 05868
[25] FENG W Z, FENG W Z, ZHANG J, et al. Graph random neural networks for semi-supervised learning on graphs[C] //Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver, Canada: ACM, 2020: 22092-22103.
[26] HAMILTON W L, HAMILTON W L, YING R, et al. Inductive representation learning on large graphs[C] //Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: ACM, 2017: 1025-1035.
[27] FENG W Z, FENG W Z, DONG Y X, et al. GRAND+: scalable graph random neural networks[C] //Proceedings of the ACM Web Conference 2022. Virtual Event, France:ACM, 2022: 3248-3258.
[28] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks [EB/OL].(2016-09-09)[2026-03-31]. https://arxiv.org/ abs/1609.02907
[29] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks [EB/OL].(2017-10-30)[2026-03-31]. https://arxiv.org/abs/1710.10903
[30] WANG Z M, CHEN J, CHEN H P. EGAT: edge-featured graph AttentionNetwork[C] //Artificial Neural Networks and Machine Learning-ICANN 2021. Cham, Switzerland: Springer, 2021: 253-264
[31] ZENG H, ZHOU H, SRIVASTAVA A, et al. GraphSAINT: graph sampling based inductive learning method [EB/OL].(2019-07-10)[2026-03-31]. https://arxiv.org/abs/1907.04931
[32] BOJCHEVSKIA, BOJCHEVSKI A, GASTEIGER J, et al. Scaling graph neural networks with approximate PageRank[C] //Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event, USA: ACM, 2020: 2464-2473.
[33] DWIVEDI V P, BRESSON X. A generalization of transformer networks to graphs [EB/OL].(2020-12-17)[2026-03-31]. https://arxiv.org/abs/2012.09699
[34] YING C, CAI T, LUO S, et al. Do transformers really perform badly for graph representation? [EB/OL].(2021-06-09)[2026-03-31]. https://10.48550/arXiv.2106.05234
[35] YING C, CAI T, LUO S, et al. Do transformers really perform badly for graph representation? [EB/OL].(2021-06-09)[2026-03-31]. https://10.48550/arXiv.2106.05234
[1] TANG Kai, WANG Fang, LIU Jianxia. Graph node classification algorithm based on multi-level core aggregation GNN [J]. Journal of Shandong University(Engineering Science), 2026, 56(3): 137-143.
[2] CHEN Yu, MENG Guangting, ZONG Chen, YUAN Weihua, WANG Jiening, WANG Xing. Algorithm for two-sided collaborative filtering multimodal contrastive representation enhancement recommender [J]. Journal of Shandong University(Engineering Science), 2026, 56(3): 106-117.
[3] DENG Bin, ZHANG Zongbao, ZHAO Wenmeng, LUO Xinhang, WU Qiuwei. Cloud-edge collaborative and graph neural network based load forecasting method for electric vehicle charging stations [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 62-69.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd