Journal of Shandong University(Engineering Science) ›› 2020, Vol. 50 ›› Issue (2): 60-65.doi: 10.6040/j.issn.1672-3961.0.2019.760

• Machine Learning & Data Mining • Previous Articles     Next Articles

LDA-based topic feature representation method for symbolic sequences

Chao FENG1,2(),Kunpeng XU1,2,Lifei CHEN1,2,*()   

  1. 1. College of Mathematics and Informatics, Fujian Normal University, Fuzhou 350117, Fujian, China
    2. Digital Fujian Internet-of-Things Laboratory of Environmental Monitoring, Fuzhou 350117, Fujian, China
  • Received:2019-12-18 Online:2020-04-20 Published:2020-04-16
  • Contact: Lifei CHEN E-mail:fc_fight2017@163.com;clfei@fjnu.edu.cn
  • Supported by:
    国家自然科学基金资助项目(61672157);国家自然科学基金资助项目(U1805263);福建师范大学创新团队资助项目(IRTL1704)

RichHTML

8

PDF (PC)

1393

Abstract:

To address the problems of high feature dimensionality and high algorithm time complexity in the existing methods, a topic feature representation method was proposed to transform the symbolic sequences into a set of topic probability vectors, based on the topic model latent Dirichlet allocation (LDA) commonly used in text mining. In the new method, each short sequence gram was considered as the shallow feature (word) of the sequence, and the topics with their probability distributions were extracted as the deep features of the sequences using the LDA model learning algorithm.Experiments were carried out on six real-world sequence sets, and compared with the existing grams-based and Markov model-based methods. The results showed that the new method improved the learning efficiency of the representation model while reducing the feature dimensionality, and achieved better accuracy in the application of symbolic sequence classification.

Key words: feature representation, symbolic sequences, latent Dirichlet allocation, topics, classification

CLC Number: 

  • TP311

Fig.1

Schematic of the SLDA model"

Table 1

Summarized parameters of the experimental datasets"

数据集 序列数M 类别数目C 平均序列长度 平均符号数目
GS1 771 8 1 594 4
GS2 281 6 1 318 4
GS3 310 6 1 536 4
SS1 50 5 1 899 15
SS2 50 5 1 498 16
SS3 50 5 925 15

Fig.2

Change of F1 with various numbers of SLDA topics"

Table 2

Comparison of the number of extracted features and F1 of the classification results yielded by the different representation methods on the symbolic sequence sets"

数据集 SLDA 特征数目 MM-FR 特征数目 G-FR 特征数目
GS1 0.946 8 35 0.911 8 16 0.980 5 81
GS2 1.000 0 15 0.996 4 16 0.996 4 85
GS3 0.967 7 35 0.922 6 16 0.996 8 72
SS1 1.000 0 15 1.000 0 324 0.980 0 227 7
SS2 1.000 0 5 1.000 0 400 1.000 0 252 9
SS3 1.000 0 5 1.000 0 400 1.000 0 212 1
1 DONG G , PEI J . Sequence data mining[M]. Berlin: Springer, 2007: Ⅶ- 69.
2 BENGIO Y , COURVILLE A , VINCENT P . Representation learning: a review and new perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intellig-ence, 2013, 35 (8): 1798- 1828.
doi: 10.1109/TPAMI.2013.50
3 COVER T M , HART P E . Nearest neighborpatternclassification[J]. IEEE Transactions on Information Theory, 1967, 13 (1): 21- 27.
4 XING Z , PEI J , KEOGH E J . A brief survey on sequence classification[J]. ACM SIGKDD Explorations Newsletter, 2010, 12 (1): 40- 48.
doi: 10.1145/1882471.1882478
5 BLASIAK S, RANGWALA H. A hidden Markov model variant for sequence classification[C]//Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Catalonia, Spain: IJCAI, 2011: 1192-1197.
6 郭彦明.基于隐马尔可夫模型的DNA序列分类研究[D].福州:福建师范大学, 2015.
GUO Yanming. A study of DNA sequence classification based on hidden Markov model[D]. Fuzhou: Fujian Normal University, 2015.
7 GUO G , CHEN L , YE Y , et al. Cluster validation method for determining the number of clusters in categorical sequences[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28 (12): 2936- 2948.
doi: 10.1109/TNNLS.2016.2608354
8 YUAN L , WANG W , CHEN L . Two-stage pruning method for gram-based categorical sequence clustering[J]. International Journal of Machine Learning and Cybernetics, 2019, 10 (4): 631- 640.
doi: 10.1007/s13042-017-0744-y
9 GERS F A , SCHMIDHUBER J , Cummins F . Learning toforget: continual prediction with LSTM[J]. Neural Computation, 2000, 12 (10): 2451- 2471.
doi: 10.1162/089976600300015015
10 GREFF K , SRIVASTAVA R K , KOUTNIK J , et al. LSTM: a search space odyssey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28 (10): 2222- 2232.
doi: 10.1109/TNNLS.2016.2582924
11 GRAVES A, MOHAMED A R, HINTON G. Speech recognition with deep recurrent neural networks[C]//Proceedings of the 2013 IEEE International Conference on Acoustics, Speech, and Signal Processing. Vancouver, Canada: IEEE, 2013: 6645-6649.
12 TANG D, QIN B, LIU T. Document modeling with gated recurrent neural network for sentiment classification[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal: ACL, 2015: 1422-1432.
13 JEBARA T , KONDOR R I , HOWARD A . Probability product kernels[J]. Journal of Machine Learning Research, 2004, 5 (5): 819- 844.
14 SALTON G . A vector space model for automatic indexing[J]. Communications of the ACM, 1975, 18 (11): 613- 620.
doi: 10.1145/361219.361220
15 XIONG T, WANG S, JIANG Q. A new Markov Model for clustering categorical sequences[C]//Proceedings of the 11th IEEE International Conference on Data Mining. Vancouver, Canada: IEEE Computer Society, 2011: 854-863.
16 程铃钫, 郭躬德, 陈黎飞. 符号序列多阶Markov分类[J]. 计算机应用, 2017, 37 (7): 1977- 1982.
CHENG Lingfang , GUO Gongde , CHEN Lifei . Classification of symbolic sequences with multi-order Markov Model[J]. Journal of Computer Applications, 2017, 37 (7): 1977- 1982.
17 郭彦明, 陈黎飞, 郭躬德. 基于隐马尔科夫模型的DNA序列分类方法[J]. 计算机系统应用, 2014, 23 (7): 24- 30.
doi: 10.3969/j.issn.1003-3254.2014.07.005
GUO Yanming , CHEN Lifei , GUO Gongde . DNA sequence classification method based on Hidden Markov Model[J]. Computer Systems & Applications, 2014, 23 (7): 24- 30.
doi: 10.3969/j.issn.1003-3254.2014.07.005
18 周玉元, 周铁军. DNA序列分类的Fisher判别法[J]. 湖南农业大学学报(自然科学版), 2003, 29 (5): 437- 440.
ZHOU Yuyuan , ZHOU Tiejun . The Fisher criterion on classification of the DNA sequence[J]. Journal of Hunan Agricultural University (Natural Sciences), 2003, 29 (5): 437- 440.
19 DAI A M , LE Q V . Semi-supervised sequence learning[J]. Advances in Neural Information Processing Systems, 2015, 3079- 3087.
20 GRAVES A, JAITLY N, MOHAMED A R. Hybrid speech recognition with deep Bidirectional LSTM[C]//Proceedings of 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. Olomouc, Czech Republic: IEEE, 2013: 273-278.
21 DEERWESTER S , DUMAIS S T , LANDAUER T K , et al. Indexing by Latent Semantic Analysis[J]. Journal of the American Society for Information Science, 1990, 41 (6): 391- 407.
doi: 10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9
22 THOMAS H. Probabilisticlatent semantic analysis[C]//Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence. Stockholm, Sweden: Morgan Kaufmann, 1999: 289-296.
23 BLEI D M , NG A Y , JORDAN M I . Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003, 3, 993- 1022.
24 GRIFFITHS T L , STEYVERS M . Finding scientific topics[J]. Proceedings of the National Academy of Sciences of the United States of America, 2004, 101 (1): 5228- 5235.
25 KELIL A, WANG S. SCS: a new similarity measure for categorical sequences[C]// Proceedings of the 8th IEEE International Conference on Data Mining. Pisa, Italy: IEEE Computer Society, 2008: 343-352.
26 WEI D , JIANG Q , WEI Y , et al. A novel hierarchical clustering algorithm for gene sequences[J]. BMC Bioinformatics, 2012, 13 (1): 174- 186.
[1] Jianqing WU,Yanqiang HUO,Jianzhu WANG,Hongyu GUO. Research review of highway differentiated toll collection [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 18-29.
[2] Hong YU,Juan DU,Lin WEI,Li ZHANG. Data fitting method for electricity consumption of power market users considering behavioral characteristics [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 113-119.
[3] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[4] Caihui LIU,Qi ZHOU,Xiaowen YE. An intrusion detection model based on improved ReliefF algorithm [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 1-10.
[5] ZHANG Qinyang, LI Xu, YAO Chunlong, LI Changwu. Aspect-level sentiment classification combined with syntactic dependency information [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 83-89.
[6] Zhuoyu XIAO,Pei HE,Guo CHEN,Yunbiao XU,Jie GUO. Design pattern classification mining with feature metrics constraints [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 48-58.
[7] HUO Bingqiang, ZHOU Tao, LU Huiling, DONG Yali, LIU Shan. Lung tumor benign-malignant classification based on multi-modal residual neural network and NRC algorithm [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 59-67.
[8] MA Changxia, ZHANG Chen. Pre-trained based joint model for intent classification and slot filling in Chinese spoken language understanding [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 68-75.
[9] ZHAO Ningning, TANG Xuesong, ZHAO Mingbo. Depth segment classification algorithm based on convolutional neural network [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 22-27.
[10] Shiqi SONG,Yan PIAO,Zexin JIANG. Vehicle classification and tracking for complex scenes based on improved YOLOv3 [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 27-33.
[11] Chunyang LI,Nan LI,Tao FENG,Zhuhe WANG,Jingkai MA. Abnormal sound detection of washing machines based on deep learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 108-117.
[12] Mingxia GAO,Jingwei LI. Chinese short text classification method based on word2vec embedding [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 34-41.
[13] Qingtao QU,Qicheng LIU,Chunxiao MU. A parallel adaptive news topic tracking algorithm based on N-Gram language model [J]. Journal of Shandong University(Engineering Science), 2018, 48(6): 37-43.
[14] Yao LI, Zhihai WANG, Yan′ge SUN, Wei ZHANG. An adaptive ensemble classification method based on deep attribute weighting for data stream [J]. Journal of Shandong University(Engineering Science), 2018, 48(6): 44-55.
[15] Pu ZHANG,Chang LIU,Yong WANG. Suggestion sentence classification model based on feature fusion and ensemble learning [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 47-54.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] SHI Lai-shun,WAN Zhong-yi . Synthesis and performance evaluation of a novel betaine-type asphalt emulsifier[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 112 -115 .
[2] LIU Wen-liang, ZHU Wei-hong, CHEN Di, ZHANG Hong-quan. Detection and tracking of moving targets using the morphology match in radar images[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 31 -36 .
[3] HAO Ranhang,CHEN Shouyu . The theory, model and method of water resources evaluationombining quantity with quality[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(3): 46 -50 .
[4] SUN Cong-zheng,GUAN Cong-sheng,QIN Jing-yu,CHENG Chuan . The structure and performances of the electroless Ni-P alloy coating on aluminum alloy[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 108 -112 .
[5] HU Tian-liang,LI Peng,ZHANG Cheng-rui,ZUO Yi . Design of a QEP decode counter based on VHDL[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 10 -13 .
[6] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 104 -107 .
[7] ZHANG Gong-xiao,YANG Rong-hua . Synthesis and characterization of salicylaldehyde methylthiosemicarbazone Schiff base complexes[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 108 -111 .
[8] CHEN Peng, HU Wen-Rong, FEI Hai-Yan. Screening of a denitrifying bacterium strain LZ-14 and its nitrogen removal characteristics[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 133 -138 .
[9] XU Yan-sheng,LIU Xing-fang . Application of the fuzzy clustering iterative model to the evalution of water resource carrying capacity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(3): 100 -104 .
[10] GAO Yang, ZHANG Qing-Song, YUAN Xiao-Shuai, XU Zhen-Hao, LIU Bin. Application of geological radar to geological forecast in karst tunnel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 82 -86 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd