Journal of Shandong University(Engineering Science) ›› 2021, Vol. 51 ›› Issue (1): 100-107.doi: 10.6040/j.issn.1672-3961.0.2020.247

• Machine Learning & Data Mining • Previous Articles     Next Articles

Unmanned vehicle path planning based on deep Q learning in real environment

Hao XIAO1(),Zhuhua LIAO1,2,*(),Yizhi LIU1,2,Silin LIU1,Jianxun LIU1,2   

  1. 1. School of Computer Science and Engineering, Hunan University of Science and Technology, Xiangtan 411201, Hunan, China
    2. Hunan Provincial Key Laboratory of Knowledge Processing and Networked Manufacturing, Xiangtan 411201, Hunan, China
  • Received:2020-06-28 Online:2021-02-20 Published:2021-03-01
  • Contact: Zhuhua LIAO E-mail:xiaohao1217@foxmail.com;zhliao@hnust.edu.cn

RichHTML

25

PDF (PC)

1323

Abstract:

It was an important problem for the intelligent navigation of unmanned vehicles that planning the optimal path in the actual traffic environment. At present, many researches about global path planning of unmanned vehicle mainly focused on the improvement of algorithm solution speed in the simulation environment. Most of them just only considered the optimal path distance or the current road conditions, also ignored other factors and future changes in the actual environment. In order to complete the complex task that competing global path planning of unmanned vehicle in dynamic road network, this research put forward a framework of unmanned vehicle driving system for practical environment based on the thought of planning after prediction, and put forward DP-DQN which was a fast global path planning method combined with deep Q learning and deep prediction network technology. This method used the road characteristic data such as time and space, weather et al to predict the future traffic situation, and then competed the global optimal path. Finally, experimental results based on open datasets showed that the proposed method reduced driving time 17.97% at most than Dijkstra, A*, algorithm et al.

Key words: global path planning, traffic environment, urban road network, deep Q learning, deep prediction network

CLC Number: 

  • TP311

Fig.1

Framework of UVS for real environment"

Fig.2

DP-DQN method framework"

Fig.3

Topology of Chengdu Road Network"

Fig.4

Road network storage conversion example diagram"

Fig.5

Example of POI clustering results"

Fig.6

Deep prediction network regression prediction result"

Table 1

Example of raw data"

字段 说明 样例数据
obj_id TTI对象id 841
batch_time 时刻 2018-01-01T00:00:00
tti 交通指数数据 1.186 65
speed 平均速度 47.398 3
geom TTI对象几何范围 MULTILINESTRING((104.137 43 30.605 91, 104.1383230.60538), …)
access_time 通行时间 10.657 6
temperature 温度 5
weather 天气 多云
wind 风力 无持续微风
AQI 空气质量指数 115轻度污染
POI经纬度 地图兴趣点经纬度(GCJ-02坐标系) 104.020 22, 30.702 2
是否点击 2: 有点击1: 未点击 2

Table 2

Evaluation of network performance"

网络类型 MAE MSE 耗时/us
DNN 5.64 7.74 17
RNN 6.97 7.85 911
GRU 5.52 6.28 2 000
LSTM 5.79 8.16 1 911

Fig.7

Deep Q learning training effect with two action space settings"

Table 3

Experimental data statistics"

算法 距离1 km 距离5 km 距离10 km 距离15 km 距离20 km 距离25 km
O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min
A* 0.49 6.12 1.56 14.89 7.32 18.37 18.34 25.10 45.58 36.01 102.04 39.51
Dijkstra 3.63 6.23 6.48 14.75 26.26 18.64 44.87 25.78 111.35 36.49 232.64 39.88
DQN 0.31 6.59 1.03 14.12 4.14 18.90 6.87 25.56 8.74 36.13 10.89 39.10
DP-DQN 0.46 5.11 1.42 13.24 5.64 16.17 7.19 21.26 10.26 32.03 18.76 37.13
ATL 0.27 6.61 1.13 13.73 3.28 19.14 9.84 25.17 12.87 35.25 25.57 38.33
DATL 1.17 5.12 3.23 13.84 6.74 15.81 13.87 21.22 17.49 32.24 31.46 37.51

Fig.8

The results of the two indicators were compared in the experiment"

1 DIKMEN M, BURNS C M. Autonomous driving in the real world: Experiences with tesla autopilot and summon[C]//Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Ann Arbor MI, USA: ACM, 2016.
2 PADEN B , CAP M , YONG S Z , et al. A survey of motion planning and control techniques for self-driving urban vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2016, 1 (1): 33- 55.
doi: 10.1109/TIV.2016.2578706
3 HART P E , NILSSON N J , RAPHAEL Bertram . A formal basis for the heuristic determination of minimum cost paths[J]. IEEE Transactions on Systems Science and Cybernetics, 1968, 4 (2): 100- 107.
doi: 10.1109/TSSC.1968.300136
4 DIJKSTRA E W . A note on two problems in connexion with graphs[J]. Numerische Mathematik, 1959, 1 (1): 269- 271.
doi: 10.1007/BF01386390
5 NIEWOLA A , PODSEDKOWSKI L . L* algorithm: a linear computational complexity graph searching algorithm for path planning[J]. Journal of Intelligent & Robotic Systems, 2018, 91 (3-4): 425- 444.
6 GOLDBERG A V, HARRELSON C. Computing the shortest path: a search meets graph theory[C]//Proceedings of the Sixteenth Annual Acm-Siam Symp-osium on Discrete Algorithms. Vancouver, Canada: SIAM, 2005.
7 PANOV A I , YAKOVLEV K S , SUVOROV R . Grid path planning with deep reinforcement learning: Preliminary results[J]. Procedia Computer Science, 2018, 123, 347- 353.
doi: 10.1016/j.procs.2018.01.054
8 DED B, KHAN S R, HASSAN K T, et al. Travel time prediction using machine learning and weather impact on traffic conditions[C]//2019 IEEE 5th International Conference for Convergence in Technology (I2CT). Pune, India: IEEE, 2019.
9 LIEBIG T , PIATKOWSKI N , BOCKERMANN C , et al. Dynamic route planning with real-time traffic predictions[J]. Information Systems, 2017, 64, 258- 265.
doi: 10.1016/j.is.2016.01.007
10 WAN J , LIU J , SHAO Z . Mobile crowd sensing for traffic prediction in internet of vehicles[J]. Sensors, 2016, 16 (1): 88.
doi: 10.3390/s16010088
11 HE F, XU Y, WANG X, et al. ALT-based route planning in dynamic time-dependent road networks[C]//Proceedings of the 2019 2nd International Conference on Machine Learning and Machine Intelligence. Jakarta, Indonesia: ACM, 2019.
12 DIAL R , GLOVER F , KARNEY D , et al. A computational analysis of alternative algorithms and labeling techniques for finding shortest path trees[J]. Networks, 1979, 9 (3): 215- 248.
doi: 10.1002/net.3230090304
13 TSO G K F , YAU K K W . Predicting electricity energy consumption: a comparison of regression analysis, decision tree and neural networks[J]. Energy, 2007, 32 (9): 1761- 1768.
doi: 10.1016/j.energy.2006.11.010
[1] Zhuoyu XIAO,Pei HE,Guo CHEN,Yunbiao XU,Jie GUO. Design pattern classification mining with feature metrics constraints [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 48-58.
[2] Wenkai ZHANG,Ke YU,Xiaofei WU. Entity recommendation based on normalized similarity measure of meta graph in heterogeneous information network [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 66-75.
[3] Chao FENG,Kunpeng XU,Lifei CHEN. LDA-based topic feature representation method for symbolic sequences [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 60-65.
[4] Delei CHEN,Cheng WANG,Jianwei CHEN,Yiyin WU. GRU-based collaborative filtering recommendation algorithm with active learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 21-27,48.
[5] Qijie ZOU,Haoyu LI,Rubo ZHANG,Tengda PEI,Yan LIU. Survey of human-robot interaction control for autonomous driving [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 23-33.
[6] Zhongwei ZHANG,Hongyan MEI,Jun ZHOU,Huiping JIA. A rule extraction method based on multi-objective co-evolutionarygenetic algorithm [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 122-130.
[7] Xiaoyan GONGYE,Peiguang LIN,Weilong REN. Genetic algorithm based on Grefenstette coding and 2-opt optimized [J]. Journal of Shandong University(Engineering Science), 2018, 48(6): 19-26.
[8] HE Dongzhi, ZHANG Jifeng, ZHAO Pengfei. Parallel implementing probabilistic spreading algorithm using MapReduce programming mode [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 22-28.
[9] DU Xixi, LIU Huafeng, JING Liping. An additive co-clustering for recommendation of integrating social network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 96-102.
[10] WANG Huan, ZHOU Zhongmei. An over sampling algorithm based on clustering [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 134-139.
[11] SHEN Ji, MA Zhiqiang, LI Tuya, ZHANG Li. A word extend LDA model for short text sentiment [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 120-126.
[12] YANG Tianpeng, XU Kunpeng, CHEN Lifei. Coefficient of variation clustering algorithm for non-uniform data [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 140-145.
[13] LI Ming, LIU Wei, ZHANG Yanduo. Mulit-Agent dynamic task allocation based on improved contract net protocol [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(2): 51-56.
[14] XIAO Zhuoyu, HE Pei, YU Bo, LI Yan, HU Zhentao. Design patterns detection based on FCA and CBR [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(2): 22-28.
[15] WANG Huiqing, SUN Hongwei, ZHANG Jianhui. Time series similarity searching algorithm based on Map/Reduce [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(1): 15-21.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] ZHANG Yong-hua,WANG An-ling,LIU Fu-ping . The reflected phase angle of low frequent inhomogeneous[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 22 -25 .
[3] SHI Lai-shun,WAN Zhong-yi . Synthesis and performance evaluation of a novel betaine-type asphalt emulsifier[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 112 -115 .
[4] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[5] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[6] LI Liang, LUO Qiming, CHEN Enhong. Graph-based ranking model for object-level search
[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 15 -21 .
[7] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[8] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[9] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[10] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 27 -32 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd