您的位置:山东大学 -> 科技期刊社 -> 《山东大学学报(工学版)》

山东大学学报 (工学版) ›› 2021, Vol. 51 ›› Issue (1): 100-107.doi: 10.6040/j.issn.1672-3961.0.2020.247

• 机器学习与数据挖掘 • 上一篇    下一篇

实际环境中基于深度Q学习的无人车路径规划

肖浩1(),廖祝华1,2,*(),刘毅志1,2,刘思林1,刘建勋1,2   

  1. 1. 湖南科技大学计算机科学与工程学院, 湖南 湘潭 411201
    2. 知识处理与网络化制造湖南省普通高校重点实验室, 湖南 湘潭 411201
  • 收稿日期:2020-06-28 出版日期:2021-02-20 发布日期:2021-03-01
  • 通讯作者: 廖祝华 E-mail:xiaohao1217@foxmail.com;zhliao@hnust.edu.cn
  • 作者简介:肖浩(1995—), 男, 四川成都人, 硕士研究生, 主要研究方向为机器学习和路径规划. E-mail: xiaohao1217@foxmail.com
  • 基金资助:
    国家科学自然基金资助项目(61370227);湖南省自然科学基金资助项目(2017JJ2081);湖南省自然科学基金资助项目(2018JJ4052);湖南省教育厅重点资助项目(17A070);湖南省教育厅重点资助项目(19A172);湖南省教育厅重点资助项目(19A174);科学研究资助项目(17C0646);科学研究资助项目(19C0755)

Unmanned vehicle path planning based on deep Q learning in real environment

Hao XIAO1(),Zhuhua LIAO1,2,*(),Yizhi LIU1,2,Silin LIU1,Jianxun LIU1,2   

  1. 1. School of Computer Science and Engineering, Hunan University of Science and Technology, Xiangtan 411201, Hunan, China
    2. Hunan Provincial Key Laboratory of Knowledge Processing and Networked Manufacturing, Xiangtan 411201, Hunan, China
  • Received:2020-06-28 Online:2021-02-20 Published:2021-03-01
  • Contact: Zhuhua LIAO E-mail:xiaohao1217@foxmail.com;zhliao@hnust.edu.cn

摘要:

实际交通环境规划最优路径的重要问题是无人车智能导航, 而无人车全局路径规划研究主要在于模拟环境中算法求解速度的提升, 考虑大部分仅路径距离最优或局限于当前道路的自身状况, 本研究针对实际环境中的其他因素及其未来的变化和动态路网中无人车全局路径规划的复杂任务, 基于预测后再规划的思想提出面向实际环境的无人车驾驶系统框架, 并结合深度Q学习和深度预测网络技术提出一种快速全局路径规划方法(deep prediction network and deep Q network, DP-DQN), 从而利用时空、天气等道路特征数据来预测未来交通状况、求解全局最优路径。基于公开数据集的试验和评价后发现, 本研究提出的方法与Dijkstra、A*等算法相比, 行车时间最高降低了17.97%。

关键词: 路径规划, 交通环境, 城市路网, 深度Q学习, 深度预测网络

Abstract:

It was an important problem for the intelligent navigation of unmanned vehicles that planning the optimal path in the actual traffic environment. At present, many researches about global path planning of unmanned vehicle mainly focused on the improvement of algorithm solution speed in the simulation environment. Most of them just only considered the optimal path distance or the current road conditions, also ignored other factors and future changes in the actual environment. In order to complete the complex task that competing global path planning of unmanned vehicle in dynamic road network, this research put forward a framework of unmanned vehicle driving system for practical environment based on the thought of planning after prediction, and put forward DP-DQN which was a fast global path planning method combined with deep Q learning and deep prediction network technology. This method used the road characteristic data such as time and space, weather et al to predict the future traffic situation, and then competed the global optimal path. Finally, experimental results based on open datasets showed that the proposed method reduced driving time 17.97% at most than Dijkstra, A*, algorithm et al.

Key words: global path planning, traffic environment, urban road network, deep Q learning, deep prediction network

中图分类号: 

  • TP311

图1

面向实际环境的无人车驾驶系统框架"

图2

DP-DQN算法框架"

图3

成都市路网拓扑图"

图4

路网存储转换示例图"

图5

POI聚类结果"

图6

深度预测网络回归预测结果示例"

表1

原始数据示例"

字段 说明 样例数据
obj_id TTI对象id 841
batch_time 时刻 2018-01-01T00:00:00
tti 交通指数数据 1.186 65
speed 平均速度 47.398 3
geom TTI对象几何范围 MULTILINESTRING((104.137 43 30.605 91, 104.1383230.60538), …)
access_time 通行时间 10.657 6
temperature 温度 5
weather 天气 多云
wind 风力 无持续微风
AQI 空气质量指数 115轻度污染
POI经纬度 地图兴趣点经纬度(GCJ-02坐标系) 104.020 22, 30.702 2
是否点击 2: 有点击1: 未点击 2

表2

定量评估网络性能"

网络类型 MAE MSE 耗时/us
DNN 5.64 7.74 17
RNN 6.97 7.85 911
GRU 5.52 6.28 2 000
LSTM 5.79 8.16 1 911

图7

2种动作空间设置的深度Q学习loss函数收敛对比"

表3

试验数据统计"

算法 距离1 km 距离5 km 距离10 km 距离15 km 距离20 km 距离25 km
O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min O(n)/ms H/min
A* 0.49 6.12 1.56 14.89 7.32 18.37 18.34 25.10 45.58 36.01 102.04 39.51
Dijkstra 3.63 6.23 6.48 14.75 26.26 18.64 44.87 25.78 111.35 36.49 232.64 39.88
DQN 0.31 6.59 1.03 14.12 4.14 18.90 6.87 25.56 8.74 36.13 10.89 39.10
DP-DQN 0.46 5.11 1.42 13.24 5.64 16.17 7.19 21.26 10.26 32.03 18.76 37.13
ATL 0.27 6.61 1.13 13.73 3.28 19.14 9.84 25.17 12.87 35.25 25.57 38.33
DATL 1.17 5.12 3.23 13.84 6.74 15.81 13.87 21.22 17.49 32.24 31.46 37.51

图8

O(n)随求解问题规模变化和生成路径的行车时间H对比"

1 DIKMEN M, BURNS C M. Autonomous driving in the real world: Experiences with tesla autopilot and summon[C]//Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Ann Arbor MI, USA: ACM, 2016.
2 PADEN B , CAP M , YONG S Z , et al. A survey of motion planning and control techniques for self-driving urban vehicles[J]. IEEE Transactions on Intelligent Vehicles, 2016, 1 (1): 33- 55.
doi: 10.1109/TIV.2016.2578706
3 HART P E , NILSSON N J , RAPHAEL Bertram . A formal basis for the heuristic determination of minimum cost paths[J]. IEEE Transactions on Systems Science and Cybernetics, 1968, 4 (2): 100- 107.
doi: 10.1109/TSSC.1968.300136
4 DIJKSTRA E W . A note on two problems in connexion with graphs[J]. Numerische Mathematik, 1959, 1 (1): 269- 271.
doi: 10.1007/BF01386390
5 NIEWOLA A , PODSEDKOWSKI L . L* algorithm: a linear computational complexity graph searching algorithm for path planning[J]. Journal of Intelligent & Robotic Systems, 2018, 91 (3-4): 425- 444.
6 GOLDBERG A V, HARRELSON C. Computing the shortest path: a search meets graph theory[C]//Proceedings of the Sixteenth Annual Acm-Siam Symp-osium on Discrete Algorithms. Vancouver, Canada: SIAM, 2005.
7 PANOV A I , YAKOVLEV K S , SUVOROV R . Grid path planning with deep reinforcement learning: Preliminary results[J]. Procedia Computer Science, 2018, 123, 347- 353.
doi: 10.1016/j.procs.2018.01.054
8 DED B, KHAN S R, HASSAN K T, et al. Travel time prediction using machine learning and weather impact on traffic conditions[C]//2019 IEEE 5th International Conference for Convergence in Technology (I2CT). Pune, India: IEEE, 2019.
9 LIEBIG T , PIATKOWSKI N , BOCKERMANN C , et al. Dynamic route planning with real-time traffic predictions[J]. Information Systems, 2017, 64, 258- 265.
doi: 10.1016/j.is.2016.01.007
10 WAN J , LIU J , SHAO Z . Mobile crowd sensing for traffic prediction in internet of vehicles[J]. Sensors, 2016, 16 (1): 88.
doi: 10.3390/s16010088
11 HE F, XU Y, WANG X, et al. ALT-based route planning in dynamic time-dependent road networks[C]//Proceedings of the 2019 2nd International Conference on Machine Learning and Machine Intelligence. Jakarta, Indonesia: ACM, 2019.
12 DIAL R , GLOVER F , KARNEY D , et al. A computational analysis of alternative algorithms and labeling techniques for finding shortest path trees[J]. Networks, 1979, 9 (3): 215- 248.
doi: 10.1002/net.3230090304
13 TSO G K F , YAU K K W . Predicting electricity energy consumption: a comparison of regression analysis, decision tree and neural networks[J]. Energy, 2007, 32 (9): 1761- 1768.
doi: 10.1016/j.energy.2006.11.010
[1] 李彩虹,方春,王志强,夏斌,王凤英. 基于超混沌同步控制的移动机器人全覆盖路径规划[J]. 山东大学学报 (工学版), 2019, 49(6): 63-72.
[2] 周风余,万方,焦建成,边钧健. 家庭陪护机器人自主充电系统研究与设计[J]. 山东大学学报 (工学版), 2019, 49(1): 55-65, 74.
[3] 张强. 核环境多关节蛇形机械臂的运动控制系统设计[J]. 山东大学学报 (工学版), 2018, 48(6): 122-131.
[4] 刘彬,张仁津. 一种采用两段粒子群优化的路径规划方法[J]. 山东大学学报(工学版), 2012, 42(1): 12-18.
[5] 严宣辉, 肖国宝*. 基于定长实数路径编码机制的移动机器人路径规划[J]. 山东大学学报(工学版), 2012, 42(1): 59-65.
[6] 陈明志1,许春耀2,陈健2,余轮2. 基于语义信息的虚拟环境路径规划[J]. 山东大学学报(工学版), 2011, 41(4): 106-112.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 王素玉,艾兴,赵军,李作丽,刘增文 . 高速立铣3Cr2Mo模具钢切削力建模及预测[J]. 山东大学学报(工学版), 2006, 36(1): 1 -5 .
[2] 张永花,王安玲,刘福平 . 低频非均匀电磁波在导电界面的反射相角[J]. 山东大学学报(工学版), 2006, 36(2): 22 -25 .
[3] 施来顺,万忠义 . 新型甜菜碱型沥青乳化剂的合成与性能测试[J]. 山东大学学报(工学版), 2008, 38(4): 112 -115 .
[4] 孔祥臻,刘延俊,王勇,赵秀华 . 气动比例阀的死区补偿与仿真[J]. 山东大学学报(工学版), 2006, 36(1): 99 -102 .
[5] 来翔 . 用胞映射方法讨论一类MKdV方程[J]. 山东大学学报(工学版), 2006, 36(1): 87 -92 .
[6] 李梁,罗奇鸣,陈恩红. 对象级搜索中基于图的对象排序模型(英文)[J]. 山东大学学报(工学版), 2009, 39(1): 15 -21 .
[7] 陈瑞,李红伟,田靖. 磁极数对径向磁轴承承载力的影响[J]. 山东大学学报(工学版), 2018, 48(2): 81 -85 .
[8] 李可,刘常春,李同磊 . 一种改进的最大互信息医学图像配准算法[J]. 山东大学学报(工学版), 2006, 36(2): 107 -110 .
[9] 季涛,高旭,孙同景,薛永端,徐丙垠 . 铁路10 kV自闭/贯通线路故障行波特征分析[J]. 山东大学学报(工学版), 2006, 36(2): 111 -116 .
[10] 浦剑1 ,张军平1 ,黄华2 . 超分辨率算法研究综述[J]. 山东大学学报(工学版), 2009, 39(1): 27 -32 .