Journal of Shandong University(Engineering Science) ›› 2025, Vol. 55 ›› Issue (6): 35-44.doi: 10.6040/j.issn.1672-3961.0.2024.114

• Machine Learning & Data Mining • Previous Articles    

University academic named entity recognition based on the fusion of multi-feature and multi-head self-attention mechanism

WANG Yuou1, YUAN Yingchun1,2*, HE Zhenxue1, HE Chen1   

  1. WANG Yuou1, YUAN Yingchun1, 2*, HE Zhenxue1, HE Chen1(1. College ofInformation Science and Technology, Hebei Agricultural University, Baoding 071001, Hebei, China;
    2. Hebei Province Key Laboratory of Agricultural Big Data, Hebei Agricultural University, Baoding 071001, Hebei, China
  • Published:2025-12-22

PDF (PC)

2

Abstract: In order to effectively solve the entity attribution and entity nesting in the academic domain of universities, a Chinese named entity recognition model Multi-feature BiGRU-MHSA-CRF(MBMC)was proposed based on multi-feature fusion. The text semantic features of the model were represented from three aspects of character, word and position to enrich the multi-dimensional semantic features of academic text. The feature vectors were fed into BiGRU(Bi-directional Recurrent Neural Network)to capture global semantic features. In order to solve the problem of entity boundary delimitation in the academic domain of higher education, the attention mechanism was improved by introducing a multi-head self-attention mechanism with Q, K, and V weight matrices and increasing the learning parameters to improve the recognition accuracy. All possible label sequences were output to the CRF, and the entity label sequence was generated by CRF decoding. The experimental results showned that the F1 value of the model reached 89.57% and 86.14% in the public dataset CoNLL2003 and the college academic domain dataset, respectively. It was better than other traditional models.

Key words: university academic, named entity recognition, Multi-feature fusion, Bi-directional recurrent neural network, Multi-head self-attention mechanism

CLC Number: 

  • TP391
[1] ZHAO Y, ZHANG A Y, LI X C, et al. RETRACTED: construction of a performance evaluation system for private higher education institutions in China based on balanced scorecard[J]. International Journal of Electrical Engineering & Education, 2023, 60(Suppl.1): 923-930.
[2] 孟平贵. 普通高等学校学生管理规定与学生工作规范化实务手册[M]. 长春: 吉林音像出版社, 2005.
[3] FIGUEROA A. Automatically generating effective search queries directly from community question-answering questions for finding related questions[J]. Expert Systems with Applications, 2017, 77: 11-19.
[4] YANG Z J, WANG Y, GAN J H, et al. Design and research of intelligent question-answering(Q&A)system based on high school course knowledge graph[J]. Mobile Networks and Applications, 2021, 26(5): 1884-1890.
[5] LI J, SUN A X, HAN J L, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(1): 50-70.
[6] YADAV V, BETHARD S. A survey on recent advances in named entity recognition from deep learning models[C] //International Conference on Computational Linguistics, Santa Fe: USA, 2018: 20-26.
[7] GOYAL A, GUPTA V, KUMAR M. Recent named entity recognition and classification techniques: a systematic review[J]. Computer Science Review, 2018, 29: 21-43.
[8] 储德平, 万波, 李红, 等. 基于ELMO-CNN-BiLSTM-CRF模型的地质实体识别[J]. 地球科学, 2021, 46: 3039-3048. CHU Deping, WAN Bo, LI Hong, et al. Geological entity recognition based on ELMO-CNN-BiLSTM-CRF model[J]. Editorial Committee of Earth Science-Journal of China University of Geosciences, 2021, 46: 3039-3048.
[9] LUO L, YANG Z H, YANG P, et al. An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition[J]. Bioinformatics, 2018, 34(8): 1381-1388.
[10] YUAN T P, QIN X Z, WEI C J. A Chinese named entity recognition method based on ERNIE-BiLSTM-CRF for food safety domain[J]. Applied Sciences, 2023, 13(5): 2849.
[11] YANG J L, LIU Y N, QIAN M H, et al. Information extraction from electronic medical records using multitask recurrent neural network with contextual word embedding[J]. Applied Sciences, 2019, 9(18): 3658.
[12] CHENG X W. Named entity recognition in the education domain based on BERT-BiLSTM-CRF-using data structures as an example[C] //2023 International Conference on Educational Knowledge and Infor-matization(EKI). 2023, Guangzhou, China: IEEE, 2023: 5-9.
[13] ZHANG J, JING S Y, HU J H. Named entity recognition method based on bert-bilstm-attention-CRF for education field[C] //2023 4th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering(ICBAIE). Hangzhou, China: IEEE, 2023: 383-388.
[14] KYUNGHYUN C, BARTVAN M, CAGLAR G, et al. Learning phrase representations using RNN Encoder-Decoder for statistical machine translation[J]. Statistics, 2014: 1724-1734.
[15] 张召武, 徐彬, 高克宁, 等. 面向教育领域的基于SVR-BiGRU-CRF中文命名实体识别方法[J]. 中文信息学报, 2022, 36(7): 114-122. ZHANG Zhaowu, XU Bin, GAO Kening, et al. SVR-BiGRU-CRF based Chinese named entity recognition for education domain[J]. Journal of Chinese Information Processing, 2022, 36(7): 114-122.
[16] 谢雪景, 谢忠, 马凯, 等. 结合BERT与BiGRU-Attention-CRF模型的地质命名实体识别[J]. 地质通报, 2023, 42(5): 846-855. XIE Xuejing, XIE Zhong, MA Kai, et al. Geological named entity recognition combined BERT and BiGRU-Attention-CRF model[J]. Geological Bulletin of China, 2023, 42(5): 846-855.
[17] XU C W, WANG F Y, HAN J L, et al. Exploiting multiple embeddings for Chinese named entity recognition[C] //Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Beijing, China: ACM, 2019: 2269-2272.
[18] 李浩, 刘永坚, 解庆, 等. 基于多层次注意力机制的远程监督关系抽取模型[J]. 计算机科学, 2019, 46(10): 252-257. LI Hao, LIU Yongjian, XIE Qing, et al. Distant supervision relation extraction model based on multi-level attention mechanism[J]. Computer Science, 2019, 46(10): 252-257.
[19] YANG X N, XIAO Y L. Named entity recognition based on BERT-MBiGRU-CRF and multi-head self-attention mechanism[C] //2022 4th International Conference on Natural Language Processing(ICNLP). Xi'an, China: IEEE, 2022: 178-183.
[20] ZHANG Q H, WU M, LV P T, et al. Research on named entity recognition of Chinese electronic medical records based on multi-head attention mechanism and character-word information fusion[J]. Journal of Intelligent & Fuzzy Systems, 2022, 42(4): 4105-4116.
[21] YANG Z W, MA J, CHEN H C, et al. Context-aware attentive multilevel feature fusion for named entity recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(1): 973-984.
[22] MA X D, YU R L, GAO C X, et al. Research on named entity recognition method of marine natural products based on attention mechanism[J]. Frontiers in Chemistry, 2023, 11: 958002.
[23] KIM J, KO Y, SEO J. A bootstrapping approach with CRF and deep learning models for improving the biomedical named entity recognition in multi-domains[J]. IEEE Access, 2019, 7: 70308-70318.
[24] 何晨, 苑迎春, 王克俭, 等. 高校学业文本命名实体识别及数据集构建研究[J]. 计算机工程与应用, 2023, 59(22): 322-328. HE Chen, YUAN Yingchun, WANG Kejian, et al. Research on college academic text named entity recognition and dataset construction[J]. Computer Engineering and Applications, 2023, 59(22): 322-328.
[1] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd