Journal of Shandong University(Engineering Science) ›› 2020, Vol. 50 ›› Issue (2): 44-49.doi: 10.6040/j.issn.1672-3961.0.2019.313

• Machine Learning & Data Mining • Previous Articles     Next Articles

A syntactic element recognition method based on deep neural network

Yanping CHEN1,2(),Li FENG1,3,*(),Yongbin QIN1,2,Ruizhang HUANG1,2   

  1. 1. School of Computer Science and Technology, Guizhou University, Guiyang 550025, Guizhou, China
    2. Data Fusion and Analysis Laboratory (Guizhou University), Guiyang 550025, Guizhou, China
    3. Guizhou Intelligent Human-Computer Interaction Engineering Technology Research Center, Guiyang 550025, Guizhou, China
  • Received:2019-06-16 Online:2020-04-20 Published:2020-04-16
  • Contact: Li FENG E-mail:ypench@gmail.com;gzu_fl931126@163.com
  • Supported by:
    国家自然科学基金联合基金重点项目(U1836205);国家自然科学基金重大研究计划项目(91746116);贵州省重大应用基础研究项目(黔科合JZ字[2014]2001);贵州省科技重大专项计划(黔科合重大专项字[2017]3002);贵州省自然科学基金(黔科合基础[2018]1035)

RichHTML

9

PDF (PC)

632

Abstract:

It was difficult to obtain structural information in Chinese sentences by the traditional feature method. To solve the problem, according to characteristics of Chinese sentence, a Bi-LSTM-Attention-CRF model was proposed based on deep neural network. A Bi-LSTM network was used to automatically extract structural information and semantic information from raw input sentences. Attention mechanism was adopted to weight abstract semantic features for classification. An optimized label sequence was output through the CRF layer. Comparing with other methods, our model could effectively identify syntactic elements in sentences. The performance reached to 84.85% in F1 score in the evaluation data sets.

Key words: syntactic elements, information extraction, deep neural network

CLC Number: 

  • TP391

Fig.1

Syntactic elementrecognition model of Bi-LSTM-Attention-CRF"

Table 1

Model performance"

Model All Type SUB ADV RAI LOC TEM
P/% R/% F1/% P/% R/% F1/% P/% R/% F1/% P/% R/% F1/% P/% R/% F1/% P/% R/% F1/%
CRF 86.61 80.34 83.36 83.75 72.93 77.97 74.35 66.06 69.96 85.79 80.21 82.91 84.33 75.30 79.56 78.61 71.20 74.73
Bi-LSTM-CRF 86.25 83.06 84.62 82.14 79.66 80.88 70.49 68.47 69.46 89.39 83.69 86.45 84.15 73.60 78.52 81.77 78.17 79.92
Bi-LSTM-Attention-CRF 86.22 83.52 84.85 82.19 78.74 80.43 71.88 71.00 71.44 87.65 81.97 87.71 86.23 86.80 86.51 81.87 75.41 78.51
1 FILLMORE C J . Toward a modern theory of case[M]. Washington D.C., USA: Department of Health, Education and Welfare, 1968.
2 吴帅, 潘海珍. 基于隐马尔可夫模型的中文分词[J]. 现代计算机(专业版), 2018, (33): 25- 28.
WU Shuai , PAN Haizhen . Chinese word segmentation based on hidden markov model[J]. Modern Computer (Professional), 2018, (33): 25- 28.
3 姚茂建, 李晗静, 吕会华. 基于马尔科夫模型的聋生阅读输入分析[J]. 北京联合大学学报, 2018, 32 (3): 86- 92.
YAO Maojian , LI Hanjing , LV Huihua . Analysis of reading input of deaf students based on markov model[J]. Journal of Beijing Union University, 2018, 32 (3): 86- 92.
4 刘晨玥, 李兵, 吴卫星. 基于罪名相关成分标注的刑事裁判文书概要信息提取[J]. 山东科技大学学报(自然科学版), 2018, 37 (4): 92- 101.
LIU Chenyue , LI Bing , WU Weixing . Extraction of summary information of criminal judgment documents based on the labeling of relevant components of charges[J]. Journal of Shandong University of Science and Technology (Natural Science Edition), 2018, 37 (4): 92- 101.
5 COHN T, BLUNSOM P. Semantic role labeling with tree conditional random fields[C]//Proceedings of the9th Conference on Computational Natural Language Learning. Stroudsburg, USA: ACM Press, 2005: 169-172.
6 YU J D , FAN X Z , PANG W B , et al. Semantic role labeling based on conditional random fields[J]. Journal of Southeast University (English Edition), 2007, 23 (3): 361- 364.
7 王臻, 常宝宝, 穗志方. 基于分层输出神经网络的汉语语义角色标注[J]. 中文信息学报, 2014, 28 (6): 56- 61.
doi: 10.3969/j.issn.1003-0077.2014.06.008
WANG Zhen , CHANG Baobao , SUI Zhifang . Chinese semantic role labeling based on hierarchical output neural network[J]. Chinese Journal of Information, 2014, 28 (6): 56- 61.
doi: 10.3969/j.issn.1003-0077.2014.06.008
8 ZHOU Jie, XU Wei. End-to-end learning of semantic role labeling using recurrent neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. Beijing, China: Association for Computational Linguistics, 2015: 1127-1137.
9 ROTH M, LAPATA M. Neural semantic role labeling with dependency path embeddings[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin, Germany: Association for computational linguistics, 2016: 1192-1202.
10 SHA Lei, JIANG Tingsong, CHANG Baobao, et al. Capturing argument relationship for Chinese semantic role labeling[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, USA: the Association for computational linguistics, 2016: 2011-2016.
11 GUO Jiang, CHE Wanxiang, WANG Haifeng, et al. A unified architecture for semantic role labeling and relation classification[C]//Proceedings of the 26th International Conference on Computational Linguistics. Osaka, Japan: the Association for Computational Linguistics, 2016: 1264-1274.
12 王瑞波, 李济洪, 李国臣, 等. 基于Dropout正则化的汉语框架语义角色识别[J]. 中文信息学报, 2017, 31 (1): 147- 154.
WANG Ruibo , LI Jihong , LI Guochen , et al. Chinese framework semantic role recognition based on dropout regularization[J]. Chinese Journal of Information, 2017, 31 (1): 147- 154.
13 HE Luheng, LEE Kenton, LEWIS Mike, et al. Deep semantic role labeling: what works and what s next[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Vancouver, Canada: Association for Computational Linguistics, 2017: 473-483.
14 MASASHI Yoshikawa, HIROSHI Noji, YUJI Matsu-moto. A* CCG Parsing with a supertag and dependency factored model[C]// Conference on Empirical Methods in Natural Language Processing. Vancouver, Canada: Association for Computational Linguistics, 2017: 277-287.
15 TAN Zhixing, WANG Mingxuan, XIE Jun, et al. Deep semantic role labeling with self-attention[J/OL]. arXiv: 1712.01586, 2017. https://arxiv.org/pdf/1712.01586.pdf.
16 STRUBELL Emma, VERGA Patrick, ANDOR Daniel., et al. Linguistically-informed self-attention for semantic role labeling[J/OL]. arXiv: 1804.08199v3[cs.CL], 2018. https://arxiv.org/pdf/1804.08199.pdf.
17 张苗苗, 刘明童, 张玉洁, 等. 融合Gate过滤机制与深度Bi-LSTM-CRF的汉语语义角色标注[J]. 情报工程, 2018, 4 (2): 45- 53.
ZHANG Miaomiao , LIU Mingtong , ZHANG Yujie , et al. Gate filtering mechanism and deep Bi-LSTM-CRF semantic role labeling in Chinese are integrated[J]. Intelligence Engineering, 2018, 4 (2): 45- 53.
18 XUE Nianwen, MARTHA Stone Palmer. Automatic semantic role labeling for Chinese verbs[C]// Proceedings of the 19th International Joint Conference on Artificial Intelligence. San Francisco, USA: Morgan Kaufmann, 2005.
19 王明轩, 刘群. 基于深度神经网络的语义角色标注[J]. 中文信息学报, 2018, 32 (2): 50- 57.
doi: 10.3969/j.issn.1003-0077.2018.02.006
WANG Mingxue , LIU Qun . Semantic role labeling based on deep neural network[J]. Chinese Journal of Information, 2018, 32 (2): 50- 57.
doi: 10.3969/j.issn.1003-0077.2018.02.006
20 YOU Liping, LIU Kaiying. Building Chinese FrameNet database[C]// Proceedings of 2005 IEEE International Conference on Natural Language Processing and Knowledge Engineering. Wuhan, China: Chinese Information Society of China, 2005: 323-328.
[1] Ningning CHEN,Jianwei ZHAO,Zhenghua ZHOU. Visual tracking algorithm based on verifying networks [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 17-26.
[2] Dongdong SHEN,Fengyu ZHOU,Mengyuan LI,Shuqian WANG,Renhe GUO. Indoor wireless positioning based on ensemble deep neural network [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 95-102.
[3] TANG Leshuang, TIAN Guohui, HUANG Bin. An object fusion recognition algorithm based on DSmT [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(1): 50-56.
[4] LIU Fan, CHEN Zehua, CHAI Jing. A new multi-focus image fusion method based on deep neural network model [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 7-13.
[5] ZHAO Zhihong 1,2, HUANG Lei ２, LIU Feng２, CHEN Zhenyu 1,2. A survey of search technologies in Deep Web [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 15-20.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] XU Xiaodan, DUAN Zhengjie, CHEN Zhongyu. The sentiment mining method based on extended sentiment dictionary and integrated features[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(6): 15 -18 .
[2] JIA Chao,ZHAO Jian-yu,XU Bang-shu,YUE Chang-cheng,LI Shu-chen . Research on rock soil liquefaction of the Qingshui railway tunnel under dynamic vibration load[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 83 -87 .
[3] CHOU Wu-Sheng, WANG Shuo. Study on the adaptive algorithm of the force reflection robotic master under large stiffness of the environment[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(1): 1 -5 .
[4] ZHANG Ning, LI Shu-Cai, LI Ming-Tian, YANG Lei. Development of a new rock similar material[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 149 -154 .
[5] CAO Gang, DONG Chao-Yang, HUANG Ji-Bao, XUE Yu-Qing. Power system inter-area oscillation damping control with FACTS devies[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 31 -36 .
[6] . H2 white noise estimation for linear continuous-time systems with delayed measurements[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 56 -61 .
[7] HAO Ming-hui,WANG Xi-ping,WANG Min,ZHOU Shen-jie .

The solution of a oneedge crack of a finite plate with the influence of  couple stress in a uniform tension field

[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 92 -95 .
[8] HUANG Yan-min1,2, ZHU Chen-fu1*, CHEN Shu-xiang2*, SONG Cui2, XU Chao2. Micro／nano-silver migration into food simulations from  micro/nano polypropylene chambers[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(2): 110 -112 .
[9] REN Xiao-Hua, CUI Zhao-Jie. Study on extracting/ reverse extracting dephenol of gasified high-concentration phenol-containing wastewater[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(1): 93 -97 .
[10] YANG Jing,YUE Qin-yan,LI Ying,LI Ren-bo,GAO Bao-yu . Application of modified activated carbon fiber in the treatment of phosphorus-containing wastewater[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 92 -95 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd