融合词汇信息与GlobalPointer的实体识别

doi:10.6040/j.issn.1672-3961.0.2022.350

摘要/Abstract

摘要： 为了提升GlobaiPointer方法的实体边界区分性能,提出一种融合词汇信息与GlobalPointer的实体识别方法。对SoftLexicon提取的词汇特征与字符相结合,采用BiLSTM网络与RoPE编码捕捉时序与相对位置信息构建全面特征,通过实体矩阵实现实体识别。对多个数据集进行试验,本研究提出的模型相较于其他基线模型,精确率、召回率、F₁均有一定的提升,Weibo数据集中F₁达到71.33%、CMeEE数据集中F₁达到63.45%,表明本研究提出的模型架构能够进一步扩充语义表征,增强识别性能。

关键词: 相对位置编码, 词汇信息, 实体识别, 特征融合, 神经网络

中图分类号:

TP39

李明键,李卫军,王海荣. 融合词汇信息与GlobalPointer的实体识别[J]. 山东大学学报 (工学版), 2024, 54(1): 91-99.

LI Mingjian, LI Weijun, WANG Hairong. Entity recognition based on lexicon information and GlobalPointer[J]. Journal of Shandong University(Engineering Science), 2024, 54(1): 91-99.

参考文献

[1] MCCALLUM A, LI W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons[C] //Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL. [S.l.] : ACM, 2003: 188-191.
[2] BIKEL D M, MILLER S, SCHWARTZ R, et al. Nymble: a high-performance learning name-finder[EB/OL].(1998-03-27)[2021-09-15].https://arxiv.org/pdf/cmp-lg/9803003.
[3] JU Z, WANG J, ZHU F. Named entity recognition from biomedical text using SVM[C] //2011 5th International Conference on Bioinformatics and Biomedical Engineering. New York, USA: IEEE, 2011: 1-4.
[4] HUANG Z, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL].(2015-08-09)[2021-07-24]. https://arxiv.org/pdf/1508.01991.
[5] DONG X, QIAN L, GUAN Y, et al. A multiclass classification method based on deep learning for named entity recognition in electronic medical records[C] //2016 New York: Scientific Data Summit(NYSDS).New York, USA: IEEE, 2016: 1-10.
[6] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[EB/OL].(2017-06-12)[2021-03-02]. https://arxiv.org/pdf/1706.03762.
[7] 曾青霞, 熊旺平, 杜建强,等. 结合自注意力的BiLSTM-CRF的电子病历命名实体识别[J]. 计算机应用与软件, 2021, 38(3): 159-162. ZENG Qingxia, XIONG Wangping, DU Jianqiang, et al. Naming entity recognition of electronic medical records based on self-attention BiLSTM-CRF[J]. Computer Application and Software, 2021, 38(3): 159-162.
[8] 司逸晨, 管有庆. 基于Transformer编码器的中文命名实体识别模型[J]. 计算机工程, 2022, 48(7): 66-72. SI Yichen, GUAN Youqing. Chinese named entity recognition model based on transformer encoder[J]. Computer Engineering, 2022, 48(7): 66-72.
[9] 罗熹, 夏先运, 安莹, 等. 结合多头自注意力机制与BiLSTM-CRF的中文临床实体识别[J]. 湖南大学学报(自然科学版), 2021, 48(4): 45-55. LUO Xi, XIA Xianyun, AN Ying, et al. Chinese clinical entity recognition combined with multi-head self-attention mechanism and BiLSTM-CRF[J]. Journal of Hunan University(Natural Science Edition), 2021, 48(4): 45-55.
[10] 王传涛, 丁林楷, 杨学鑫, 等. 基于BERT的中文电子简历命名实体识别[J]. 中国科技论文, 2021, 16(7): 770-775. WANG Chuantao, DING Linkai, YANG Xuexin, et al. Chinese electronic resume named entity recognition based on BERT[J]. Chinese Science and Technology Paper, 2021, 16(7): 770-775.
[11] 郭军成, 万刚, 胡欣杰, 等. 基于BERT的中文简历命名实体识别[J]. 计算机应用, 2021, 41(增刊1): 15-19. GUO Juncheng, WAN Gang, HU Xinjie, et al. Chinese resume named entity recognition based on BERT[J]. Computer Application, 2021, 41(Suppl.1): 15-19.
[12] ZHANG Y, YANG J. Chinese NER using lattice LSTM[EB/OL].(2018-05-05)[2021-05-06].https://arxiv. org/pdf/1805.02023.
[13] MA R, PENG M, ZHANG Q, et al. Simplify the usage of lexicon in chinese NER[C] //Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle, USA: ACL, 2020: 5951-5960.
[14] 赵耀全, 车超, 张强. 基于新词发现和Lattice-LSTM的中文医疗命名实体识别[J]. 计算机应用与软件, 2021, 38(1): 161-165. ZHAO Yaoquan, CHE Chao, ZHANG Qiang. Chinese medical named entity recognition based on neologism discovery and Lattice-LSTM [J]. Computer Application and Software, 2021, 38(1): 161-165.
[15] GRAVES A, MOHAMED A, HINTON G. Speech recognition with deep recurrent neural networks[C] //2013 IEEE International Conference on Acoustics, Speech and Signal Processing. [S.l.] : IEEE, 2013: 6645-6649.
[16] DEVLIN J, CHANG M W, LEE K, et al. Bert: pre-training of deep bidirectional transformers for language understanding[EB/OL].(2018-10-11)[2021-05-03]. https://arxiv.org/pdf/1810.04805.
[17] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL].(2013-01-16)[2021-12-18]. http://arxiv. org/abs/1301.3781.
[18] PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C] //Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2014: 1532.
[19] SU J L, MURTADHA A, PAN S, et al. GlobalPointer: novel efficient span-based approach for named entity recognition[EB/OL].(2022-08-15)[2022-09-12]. https://arxiv.org/abs/2106.08087.
[20] WANG Y, YU B, ZHANG Y, et al. TPLinker: single-stage joint extraction of entities and relations through token pair linking[C] //Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain: International Committee on Computational Linguistics, 2020: 1572-1582.
[21] SU J L, LU Y, PAN S, et al. Roformer: enhanced transformer with rotary position embedding[EB/OL].(2021-04-20)[2021-12-01]. https://arxiv.org/pdf/2104. 09864.
[22] SU J L, ZHU M, MURTADHA A, et al. ZLPR: a novel loss for multi-label classification[EB/OL].(2022-08-05)[2022-09-12]. https://arxiv.org/pdf/2208.02955.
[23] PENG N, DREDZE M. Improving named entity recognition for chinese social media with word segmentation representation learning[EB/OL].(2016-03-02)[2021-12-05].https://arxiv.org/pdf/1603. 00786.
[24] ZHANG N, CHEN M, BI Z, et al. Cblue: a chinese biomedical language understanding evaluation benchmark[EB/OL].(2021-06-15)[2021-12-24].https://arxiv.org/pdf/2106.08087.
[25] LI X, YAN H, QIU X, et al. FLAT: Chinese NER using flat-lattice transformer[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, France: Association for Computational Linguistics, 2020: 6836-6842.
[26] 毛明毅, 吴晨, 钟义信, 等. 加入自注意力机制的BERT命名实体识别模型[J]. 智能系统学报, 2020, 15(4): 772-779. MAO Mingyi, WU Chen, ZHONG Yixin, et al. BERT named entity recognition model with self-attention mechanism[J]. Journal of Intelligent Systems, 2020, 15(4): 772-779.
[27] 李健, 熊琦, 胡雅婷, 等. 基于Transformer和隐马尔科夫模型的中文命名实体识别方法[J]. 吉林大学学报(工学版),2023, 53(5):1427-1434. LI Jian, XIONG Qi, HU Yating, et al. Chinese named entity recognition method based on Transformer and hidden Markov model[J]. Journal of Jilin University(Engineering Edition), 2023, 53(5):1427-1434.
[28] 钟诗胜, 陈曦, 赵明航, 等. 引入词集级注意力机制的中文命名实体识别方法[J]. 吉林大学学报(工学版), 2022, 52(5): 1098-1105. ZHONG Shisheng, CHEN Xi, ZHAO Minghang, et al. Chinese named entity recognition method based on word set level attention mechanism[J]. Journal of Jilin University(Engineering Edition), 2022, 52(5): 1098-1105.

相关文章 15

[1]	范黎林,刘士豪,李源,毛文涛,陈宗涛. 基于课程正则化的物理信息神经网络渐进式训练策略[J]. 山东大学学报 (工学版), 2024, 54(1): 11-24.
[2]	孙园,曾惠权,欧阳苏建,高佳倩,王绮楠,林智勇. 基于粒子群算法的模糊大脑情感学习非线性系统辨识[J]. 山东大学学报 (工学版), 2024, 54(1): 25-32.
[3]	迟云浩,杨璐,郭杰,郝凡昌,聂秀山. 基于注意力特征融合网络的手指静脉图像质量评价方法[J]. 山东大学学报 (工学版), 2023, 53(6): 56-62.
[4]	李源,张妮,张艳娜,刘士豪,李学辉. 用于预测边界元弱奇异积分的新型樽海鞘-神经网络模型[J]. 山东大学学报 (工学版), 2023, 53(6): 8-15.
[5]	陈雷,赵耀帅,林彦,郭晟楠,万怀宇,林友芳. 交通流量预测的时间异质性图注意力网络[J]. 山东大学学报 (工学版), 2023, 53(5): 29-36.
[6]	那绪博,张莹,李沐阳,陈元畅,华云鹏. 基于ODCG的网约车需求预测模型[J]. 山东大学学报 (工学版), 2023, 53(5): 48-56.
[7]	韦修喜,陶道,黄华娟. 改进果蝇算法优化BP神经网络预测汽油辛烷值[J]. 山东大学学报 (工学版), 2023, 53(5): 20-28.
[8]	范海雯,郝旭东,赵康,邢法财,蒋哲,李常刚. 基于卷积神经网络的含分布式光伏配电网静态等值[J]. 山东大学学报 (工学版), 2023, 53(4): 140-148.
[9]	于泓,杜娟,魏琳,张利. 计及行为特征的市场化用户电量数据拟合方法[J]. 山东大学学报 (工学版), 2023, 53(4): 113-119.
[10]	李浩源,于景明,张桂林,张斌. 基于智能算法的光纤预制棒芯层制备工艺参数优化[J]. 山东大学学报 (工学版), 2023, 53(4): 149-156.
[11]	宋佳芮,陈艳平,王凯,黄瑞章,秦永彬. 基于Affix-Attention的命名实体识别语义补充方法[J]. 山东大学学报 (工学版), 2023, 53(2): 70-76.
[12]	刘行,杨璐,郝凡昌. 基于多特征融合的手指静脉图像检索方法[J]. 山东大学学报 (工学版), 2023, 53(2): 118-126.
[13]	袁钺,王艳丽,刘勘. 基于空洞卷积块架构的命名实体识别模型[J]. 山东大学学报 (工学版), 2022, 52(6): 105-114.
[14]	王智伟,徐海超,郭相阳,马炯,褚云龙,陈前昌,卢治. 基于卷积神经网络和层次分析的新能源电源调频能力智能预测方法[J]. 山东大学学报 (工学版), 2022, 52(5): 70-76.
[15]	孟祥飞,张强,胡宴才,张燕,杨仁明. 欠驱动船舶自适应神经网络有限时间跟踪控制[J]. 山东大学学报 (工学版), 2022, 52(4): 214-226.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed