JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2016, Vol. 46 ›› Issue (6): 40-47.doi: 10.6040/j.issn.1672-3961.1.2016.213

Previous Articles     Next Articles

The time series prediction model based on integrated deep learning

HE Zhengyi1,2, ZENG Xianhua1,2*, QU Shengwei1,2, WU Zhilong1   

  1. HE Zhengyi1, 2, ZENG Xianhua1, 2*, QU Shengwei1, 2, WU Zhilong1(1. College of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
    2. Chongqing Key Laboratory of Computational Intelligence, Chongqing 400065, China
  • Received:2016-03-31 Online:2016-12-20 Published:2016-03-31

Abstract: The conditional restricted Boltzmann machine time series model based on the Gaussian process(GCRBM)could efficiently predict single type of time series data, but the model could not make accurate predictions for multi-category data and real high-dimensional data. To solve the problem above, the time series prediction model based on integrated deep learning was proposed. Multiple deep belief networks(DBN)corresponding to the multi-category timing data was trained to study low dimensional feature. The low dimensional feature of multi-category data was used to train multiple GCRBM models. When the time series was predicted, the dimensionality of the model was reduced and categories of target data were identified by DBN model's reconstruction error, and the sequence of target data was predicted by the GCRBM model. The experimental results based on CASIA-A gait data set showed that the method could accurately recognize the categories of gait sequences and the predicting result could simulate the true gait sequences, which demonstrated the validity of the model.

Key words: time series, prediction model, conditional restricted Boltzmann machine(GCRBM), integrated deep learning, deep belief networks(DBN)

CLC Number: 

  • TP181
[1] TAYLOR G W, HINTON G E, ROWEIS S T. Modeling human motion using binary latent variables[C] // Advances in Neural Information Processing Systems(NIPS 19): Proceedings of the 2006 Conference. [S.l.] : MIT Press, 2007: 1345-1352.
[2] TAYLOR G W, HINTON G E, ROWEIS S T. Two distributed-state models for generating high-dimensional time series[J]. Journal of Machine Learning Research, 2011, 12(2):1025-1068.
[3] HINTONG E. Training products of experts by minimizing contrastive divergence[J]. Neural Computation, 2006, 14(8):1771-1800.
[4] HINTON G E, OSINDERO S, WELLING M, et al. Unsupervised discovery of nonlinear structure using contrastive back propagation[J]. Cognitive Science, 2006, 30(4):725-731.
[5] ZENG Xianhua, LUO Siwei, WANG Jiao. Auto-associative neural network system for recognition[C] // Proceedings of International Conference on Machine Learning and Cybernetics(ICMLC 2007). [S.l.] : IEEE Conference Publications, 2007: 2885-2890.
[6] HINTON G E, SALAKHUTDINOV R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786):504-507.
[7] HINTONG G E, SALAKHUTDINOV R. Supporting online material for reducing the dimensionality of data with neural networks[J]. Science, 2006, 504(5786):504-507.
[8] HINTONG G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
[9] SMOLENSKY P. Parallel distributed processing: information processing in dynamical systems: foundations of harmony theory[M]. Cambridge, USA: MIT Press, 1986: 194-281.
[10] SALAKHUTDINOV R, JOSHUA B, TORRALBA A. Learning with hierarchical-deep models[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2013, 35(8):1958-1971.
[11] LAROCHELLE H, BEMGIO Y, LOURADOUR J, et al. Exploring strategies for training deep neural networks[J]. Journal of Machine Learning Research, 2009, 10(6):1-40.
[12] 孙志军, 薛磊, 许阳明. 深度学习研究综述[J]. 计算机应用研究, 2012, 29(8):2806-2810. SUN Zhijun, XUE Lei, XU Yangming. The research summary of deep learning[J]. Application Research of Computers, 2012, 29(8):2806-2810.
[13] 钟颖, 汪秉文. 基于遗传算法的BP神经网络时间序列预测模型[J]. 系统工程与电子技术, 2002, 24(4):9-11. ZHONG Ying, WANG Bingwen. The back propagation neural network based on genetic algorithm for predicting time series[J]. Journal of Systems Engineering and Electronics, 2002, 24(4):9-11.
[14] 张玉瑞, 陈剑波. 基于RBF神经网络的时间序列预测[J]. 计算机工程与应用, 2005, 41(11):74-76. ZHANG Yurui, CHEN Jianbo. A predicting time series model based on radial basis function neural network[J]. Computer Engineering and Applications, 2005, 41(11):74-76.
[15] 周若愚. 基于SVR与半监督学习的时间序列预测[D].西安:西安电子科技大学, 2014. ZHOU Ruoyu. A predicting time series model based on support vector regression and semi supervised learning[D]. Xi'an: Xidian University, 2014.
[1] Ming YANG, Pingjing DU, Fengquan LIU, Xupeng HAO, Yifan BO. Review of energy consumption and demand forecasting methods [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 56-62.
[2] Diankun ZHENG,Tongle XU,Zhaojie YIN,Qingmin MENG. Prediction method of tailing dam groundwater levels based on improved PSO-BP neural network [J]. Journal of Shandong University(Engineering Science), 2019, 49(3): 108-113.
[3] Shizhan SONG,Chuanyong WANG,Wenwen KANG,Jian ZHANG,Honghua YAN,Peng LI. Wind and PV installed capacity optimization with hybrid uncertainty of renewable energy [J]. Journal of Shandong University(Engineering Science), 2018, 48(6): 101-108.
[4] Shizhan SONG,Chuanyong WANG,Wenwen KANG,Jian ZHANG,Honghua YAN,Peng LI. PV installed capacity planning for power distribution network based on time series production simulation [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 131-136.
[5] HE Zhengyi, ZENG Xianhua, GUO Jiang. An ensemble method with convolutional neural network and deep belief network for gait recognition and simulation [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 88-95.
[6] XIE Guohui, FAN Hao. Prediction model of concentrating solar power technology maturity [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(6): 83-88.
[7] TAO Zhiwei, ZHANG Li. Time series classification using piecewise vector quantized approximation based on Mahalanobis distance [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 51-57.
[8] WANG Huiqing, SUN Hongwei, ZHANG Jianhui. Time series similarity searching algorithm based on Map/Reduce [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(1): 15-21.
[9] MA Chang-hui1, FENG Jiang-xia2, JIANG Zhe1, WU Nai-hu1, L Xiao-lu3. Wind power prediction based on time-series and BP-ANN [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(1): 85-89.
[10] ZHU Yue-long, LI Shi-jin, FAN Qing-song, WAN Ding-sheng. Wavelet-neural network model based complex hydrological time series prediction [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(4): 119-124.
[11] LI Xiao-bin1, LI Shi-yin2. Ensemble learning of multi-classifier for early classification of time series [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(4): 73-78.
[12] LUO Yu-Pan, SHANG Lin. Detect outliers in time series data with multi-granule periodic patterns [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 11-15.
[13] LIAO Huo-mu,DONG Zeng-chuan,SHU Long-cang,YUN Ru-an . Combinative time series analysis method for the prediction  of the groundwater level [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 96-100 .
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[2] SHI Lai-shun,WAN Zhong-yi . Synthesis and performance evaluation of a novel betaine-type asphalt emulsifier[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 112 -115 .
[3] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[4] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[5] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[6] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[7] WANG Bo,WANG Ning-sheng . Automatic generation and combinatory optimization of disassembly sequence for mechanical-electric assembly[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 52 -57 .
[8] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[9] ZHANG Ying,LANG Yongmei,ZHAO Yuxiao,ZHANG Jianda,QIAO Peng,LI Shanping . Research on technique of aerobic granular sludge cultivationby seeding EGSB anaerobic granular sludge[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(4): 56 -59 .
[10] Yue Khing Toh1, XIAO Wendong2, XIE Lihua1. Wireless sensor network for distributed target tracking: practices via real test bed development[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 50 -56 .