Journal of Shandong University(Engineering Science) ›› 2021, Vol. 51 ›› Issue (2): 105-114.doi: 10.6040/j.issn.1672-3961.0.2020.233

Previous Articles     Next Articles

Dynamic prediction of spatiotemporal big data based on relationship transfer and reinforcement learning

ZHENG Zijun1,2, FENG Xiang1,2*, YU Huiqun1,2, LI Xiuquan3   

  1. 1. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China;
    2. Shanghai Engineering Research Center of Smart Energy, Shanghai 200237, China;
    3. China Academy of Science and Technology for Development, Beijing 100038, China
  • Published:2021-04-16

Abstract: A dynamic prediction algorithm based on relationship transfer and reinforcement learning was proposed to alleviate the problem that the dynamic prediction in a large spatiotemporal range fails to obtain an accurate solution. The algorithm adopted a crowd intelligence computing manner with complex workflow models to solve the spatiotemporal data optimization problem. A relationship transfer block was designed to learn the probability of relationship transfer by extracting features from spatiotemporal data. A prediction reinforcement learning block was established along with the time series to process the transition relationship probability in parallel and prioritize the spatiotemporal data according to feature preferences that predict the problem status trend. A deep multi-step iterative strategy optimization was adopted to obtain a reasonable solution. Theoretical analysis and discussion of the convergence and convergence rate of the proposed algorithm were conducted. Experimental results on patent transfer data verified this approach's strengths and demonstrated that the ranking accuracy could be significantly improved by applying the relationship transfer block and prediction reinforcement learning block.

Key words: spatiotemporal data, complex workflow, relationship transfer, feature learning, reinforcement learning

CLC Number: 

  • TP391
[1] LU H, SU S, TIAN Z, et al. A novel search engine for Internet of everything based on dynamic prediction[J]. China Communications, 2019, 16(3): 42-52.
[2] ZHANG H, HU B, WANG X, et al. An action dependent heuristic dynamic programming approach for algal bloom prediction with time-varying parameters[J]. IEEE Access, 2020, 8: 26235-26246.
[3] ROZENSHTEIN P, GIONIS A. Temporal pagerank[C] //Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham, Switzerland: Springer, 2016: 674-689.
[4] SUN C, BAI L, KANG L, et al. An approach for predicting uncertain spatiotemporal XML data integrated with grey dynamic model[J]. IEEE Access, 2018, 6: 46801-46825.
[5] MOHAMMADI M, Al-FUQAHA A. Exploiting the spatio-temporal patterns in IoT data to establish a dynamic ensemble of distributed learners[J]. IEEE Access, 2018, 6: 63316-63328.
[6] PAGE L, BRIN S, MOTWANI R, et al. The PageRank citation ranking: Bringing order to the web[R]. Stanford, USA: Stanford InfoLab, 1999.
[7] HSU C, LAI Y, CHEN W, et al. Unsupervised ranking using graph structures and node attributes[C] //Proc-eedings of the Tenth ACM International Conference on Web Search and Data Mining. New York, USA: ACM, 2017: 771-779.
[8] GUO T, CAO X, CONG G, et al. Distributed algorithms on exact personalized pagerank[C] //Proceedings of the 2017 ACM International Conference on Management of Data. New York, USA: ACM, 2017: 479-494.
[9] CIPOLLA S, REDIVO-ZAGLIA M, TUDISCO F. Extrapolation methods for fixed-point multilinear PageRank computations[J]. Numerical Linear Algebra with Applications, 2020, 27(2): e2280.
[10] YANG X, WANG Q. Crowd hybrid model for pedestrian dynamic prediction in a corridor[J]. IEEE Access, 2019, 7: 95252-95261.
[11] GHOSH B, ASIF M, DAUWELS J, et al. Dynamic prediction of the incident duration using adaptive feature set[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(11): 4019-4031.
[12] MICHELUCCI P, DICKINSON J. The power of crowds[J]. Science, 2016, 351(6268): 32-33.
[13] NGUYEN Q, TUDISCO F, GAUTIER A, et al. An efficient multilinear optimization framework for hypergraph matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 39(6): 1054-1075.
[14] LIN W. Distributed algorithms for fully personalized pagerank on large graphs[C] //The World Wide Web Conference. New York, USA: ACM, 2019:1084-1094.
[15] WANG W, MAZAITIS K, COHEN W. Programming with personalized pagerank: a locally groundable first-order probabilistic logic[C] //Proceedings of the 22nd ACM international conference on Information & Knowledge Management. New York, USA: ACM, 2013: 2129-2138.
[16] SAEZ T, HOGAN A. Automatically generating Wikipedia info-boxes from Wikidata[C] //Companion Proceedings of the The Web Conference 2018. Lyon, France: International World Wide Web Conferences Steering Committee. Republic and Canton of Geneva, Switzerland, 2018: 1823-1830.
[17] WANG J, WANG X, WU J. Inferring metapopulation propagation network for intra-city epidemic control and prevention[C] //Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, USA: ACM, 2018: 830-838.
[18] 马建红,张明月,赵亚男. 面向创新设计的专利知识抽取方法[J]. 计算机应用, 2016, 36(2): 465-471. MA Jianhong, ZHANG Mingyue, ZHAO Ya'nan. Patent knowledge extraction method for innovation design[J]. Journal of Computer Applications, 2016, 36(2): 465-471.
[19] LI J. Exploring the logic and landscape of the knowledge system: multilevel structures, each multiscaled with complexity at the mesoscale[J]. Engineering, 2016, 2(3): 276-285.
[20] GLEICH D, LIM L, YU Y. Multilinear pagerank[J]. SIAM Journal on Matrix Analysis and Applications, 2015, 36(4): 1507-1541.
[1] GAO Junjian, LIAO Zhuhua, LIU Yizhi, ZHAO Yijiang. Hierarchical multi-agent reinforcement learning based route guidance method combining personalization and signal control [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 34-45.
[2] Junsan ZHANG,Qiaoqiao CHENG,Yao WAN,Jie ZHU,Shidong ZHANG. MIRGAN: a medical image report generation model based on GAN [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 9-18.
[3] Zhifu CHANG,Fengyu ZHOU,Yugang WANG,Dongdong SHEN,Yang ZHAO. A survey of image captioning methods based on deep learning [J]. Journal of Shandong University(Engineering Science), 2019, 49(6): 25-35.
[4] SHEN Jing, LIU Hai-bo, ZHANG Ru-bo, WU Yan-xia, CHENG Xiao-bei. Multi-robot hierarchical reinforcement learning based on semi-Markov games [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(4): 1-7.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[3] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[4] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[5] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[6] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[7] WANG Bo,WANG Ning-sheng . Automatic generation and combinatory optimization of disassembly sequence for mechanical-electric assembly[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 52 -57 .
[8] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[9] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[10] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 27 -32 .