您的位置：山东大学 -> 科技期刊社 -> 《山东大学学报（工学版）》

山东大学学报 (工学版) ›› 2021, Vol. 51 ›› Issue (1): 1-10.doi: 10.6040/j.issn.1672-3961.0.2020.382

• 控制科学与工程 •    下一篇

多运动视觉里程计的方法与技术

周风余(),顾潘龙*(),万方,尹磊,贺家凯   

  1. 山东大学控制科学与工程学院，山东 济南 250061
  • 收稿日期:2020-09-15 出版日期:2021-02-20 发布日期:2021-03-01
  • 通讯作者: 顾潘龙 E-mail:zhoufengyu@sdu.edu.cn;gupanlongly@163.com
  • 作者简介:周风余，1969年3月生，工学博士，教授，博士生导师，山东省急需紧缺高层次人才，国家重点研发计划项目首席科学家，中国自动化学会高级会员，建筑机器人专业委员会委员，IEEE机器人与自动化学会会员，山东省自动化学会科普与创新工作委员会主任。主要从事智能机器人、机器人云服务平台及智能控制理论及方法等方面的教学、研究与开发工作。周风余(1969—)，男，山东沂水人，教授，博士生导师, 主要研究方向为智能机器人和云机器人. E-mail: zhoufengyu@sdu.edu.cn
  • 基金资助:
    国家重点研发计划资助项目(2017YFB1302400);国家自然科学基金资助项目(61773242);国家自然科学基金资助项目(61803227);山东省农业重大应用技术创新资助项目(SD2019NJ014);智能机器人与系统高精尖创新中心开放研究基金资助项目(2019IRS19)

Overview of multi-motion vision odometer

Fengyu ZHOU(),Panlong GU*(),Fang WAN,Lei YIN,Jiakai HE   

  1. School of Control Science and Engineering, Shandong University, Jinan 250061, Shandong, China
  • Received:2020-09-15 Online:2021-02-20 Published:2021-03-01
  • Contact: Panlong GU E-mail:zhoufengyu@sdu.edu.cn;gupanlongly@163.com

RichHTML

397

PDF (PC)

618

摘要：

多运动视觉里程计(multi-motion visual odometry, MVO)是在动态场景中估计动态物体位姿变化的算法，对于移动机器人和自动驾驶汽车等自主化设备(autonomous things, AuT)的研发具有重要的理论意义和较大的实用价值。综述机器人领域多运动视觉里程计的发展过程及最新研究进展，根据特征点聚类方法的不同，从特征点几何聚类和语义几何聚类两个方面介绍多运动视觉里程计融合特征点及语义信息求解空间内多运动视觉里程计的重要研究成果。基于相同的评价指标及数据集对几种常用方法进行深入的对比研究，展望未来多运动视觉里程计的发展方向。

关键词: 视觉里程计, 多运动视觉里程计, 视觉同步定位与建图, 动态剔除, 帧间估计

Abstract:

Multi-motion visual odometry (MVO) was an algorithm for estimating the pose change of dynamic objects in dynamic scenes. It was of great theoretical significance and practical value in autonomous things (AuT). The development process and the latest research progress of multi-motion visual odometer in robot field were reviewed. The important research results of multi-motion visual odometer with the fusion of semantic and geometric features were introduced. Based on the same evaluation criteria and datasets, this research compared several common methods, and prospected the future development direction of multi motion visual odometer.

Key words: visual odometer, MVO, visual simultaneous localization and mapping, dynamic culling, interframe estimation

中图分类号: 

  • TP391.4

图1

MVO计算框架"

图2

几何信息优化图像分割效果"

图3

3D包围框到2D包围框的投影关系"

图4

本研究评测试用的数据集"

表1

TUM数据集定位绝对轨迹均方根误差"

子数据集名称 算法名称
ORB-SLAM2 DynaSLAM 文献[12]算法 Dynamic-SLAM MaskFusion
fr2_desk_ps 4.971 0.8 1.873
fr3_sit_half 1.992 1.62 1.461 5.2
fr3_sit_rpy 5.811 3.448
fr3_sit_xyz 0.836 1.3 0.9 0.601 3.1
fr3_walk_half 3.714 2.1 1.99 2.139 10.6
fr3_walk_rpy 10.528 2.999 6.025
fr3_walk_xyz 1.431 1.4 1.39 1.324 10.4

表2

TUM数据集定位相对平移均方根误差"

子数据集名称 算法名称
ORB-SLAM2 DVO 文献[12]算法 Dynamic-SLAM BaMVO MaskFusion
fr2_desk_ps 2.584 2.96 0.051 1.958 2.96
fr3_sit_half 1.694 10.05 1.57 1.451 5.89 4.1
fr3_sit_rpy 6.495 17.35 4.303 18.72
fr3_sit_xyz 0.948 4.53 1.37 0.998 4.82 4.6
fr3_walk_half 3.795 26.28 2.192 17.38 9.3
fr3_walk_rpy 6.776 40.38 4.48 5.605 35.84
fr3_walk_xyz 1.772 54.01 2.02 1.796 23.26 9.7

表3

TUM数据集定位相对旋转均方根误差"

子数据集名称 算法名称
ORB-SLAM2 DVO 文献[12]算法 Dynamic-SLAM BaMVO MaskFusion
fr2_desk_ps 0.891 1.392 0 0.356 1 0.833 1.116 7
fr3_sit_half 0.581 4.464 9 0.921 0 0.551 2.880 4 2.07
fr3_sit_rpy 1.368 6.016 4 0.991 5.983 4
fr3_sit_xyz 0.557 1.498 0 0.509 1 0.613 1.388 5 1.25
fr3_walk_half 0.763 5.217 9 0.666 4.286 3 3.35
fr3_walk_rpy 1.278 7.066 2 0.954 3 1.149 6.339 8
fr3_walk_xyz 1.825 7.666 9 0.541 5 0.598 4.391 1 2.0

表4

KITTI数据集上的试验结果测评"

KITTI序列 ORB-SLAM2 DynaSLAM Cluster SLAM Cluster Vo 文献[37]
ATE/ cm RRE/ cm RPE/ (°) ATE/ cm RRE/ cm RPE/ (°) ATE/ cm RRE/ cm RPE/ (°) ATE/ cm RRE/ cm RPE/ (°) ATE/ cm
0926_0009 0.91 0.01 1.89 7.51 0.06 2.17 0.92 0.03 2.34 0.79 0.03 2.98 1.14
0926_0013 0.30 0.01 0.94 1.97 0.04 1.41 2.12 0.07 5.50 0.26 0.01 1.16 0.35
0926_0014 0.56 0.01 1.15 5.98 0.09 2.73 0.81 0.03 2.24 0.48 0.01 1.04 0.51
0926_0051 0.37 0.00 1.10 10.95 0.10 1.65 1.19 0.03 1.44 0.81 0.02 2.74 0.76
0926_0101 3.42 0.03 14.27 10.24 0.13 12.29 4.02 0.02 12.43 3.18 0.02 12.78 5.30
0929_0004 0.44 0.01 1.22 2.59 0.02 2.03 1.12 0.02 2.78 0.40 0.02 1.77 0.40
1003_0047 18.87 0.05 28.32 9.31 0.05 6.58 10.21 0.06 8.94 4.79 0.05 6.54 1.03

图5

MVO与Cluster VO精度差"

1 丁文东, 徐德, 刘希龙, 等. 移动机器人视觉里程计综述[J]. 自动化学报, 2018, 44 (3): 385- 400.
DING Wendong , XU De , LIU Xilong , et al. Overview of mobile robot visual odometer[J]. Acta Automatica Sinica, 2018, 44 (3): 385- 400.
2 CADENA Caser , CARLONE Luca , CARRILLO Henry , et al. Past, present, and future of simultaneous locali-zation and mapping: toward the robust-perception age[J]. IEEE Transactions on Robotics, 2016, 32 (6): 1309- 1332.
doi: 10.1109/TRO.2016.2624754
3 LERREL Pinto, ABHINAV Gupta. Supersizing self-supervision: learning to grasp from 50k tries and 700 robot hours[C]//International Conference on Robotics and Automation (ICRA). Stockholm, Sweden: IEEE, 2016: 3406-3413.
4 张峻宁, 苏群星, 刘鹏远, 等. 一种自适应特征地图匹配的改进VSLAM算法[J]. 自动化学报, 2019, 45 (3): 553- 565.
ZHANG Junning , SU Qunxing , LIU Pengyuan , et al. An improved vSLAM algorithm for adaptive feature map matching[J]. Acta Automatica Sinica, 2019, 45 (3): 553- 565.
5 NAM Dinh Van , KIM Gon-Woo . Robust stereo visual inertial navigation system based on multi-stage outlier removal in dynamic environments[J]. Sensors, 2020, 20 (10): 2922- 2946.
doi: 10.3390/s20102922
6 MATTHIES Larry , STEVEN Shafer . Error mode-lling in stereo navigation[J]. IEEE Journal on Robotics and Automation, 1987, 3 (3): 239- 248.
doi: 10.1109/JRA.1987.1087097
7 BERTA Bescos , JOSE Facil , JAVIER Civera , et al. DynaSLAM: tracking, mapping and inpainting in dynamic scenes[J]. IEEE Robotics & Automation Letters, 2018, 32 (11): 1- 1.
8 RAúL Mur-Artal , TARDOS Juan . ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras[J]. IEEE Transactions on Robotics, 2016, 33 (5): 1255- 1262.
9 HE Kaiming, GEORGIA Gkioxari, DOLLAR Piotr, et al. Mask r-cnn[C]//International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017: 2961-2969.
10 GEIGER A , LENZ P , STILLER C , et al. Vision meets robotics: the kitti dataset[J]. The International Journal of Robotics Research, 2013, 32 (11): 1231- 1237.
doi: 10.1177/0278364913491297
11 STURM Jrgen, ENGELHARD Nikolas, ENDRES Felix, et al. A benchmark for the evaluation of RGB-D SLAM systems[C]//International Conference on Intelligent Robots and Systems. Vilamoura, Portual: IEEE, 2012: 573-580.
12 ZHAO Lili , LIU Zhili , CHEN Jianwen , et al. A compatible framework for RGB-D SLAM in dynamic scenes[J]. IEEE Access, 2019, 7 (1): 75604- 75614.
13 JOHN Canny . A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8 (6): 679- 698.
14 BOUGUET J Y . Pyramidal implementation of the affine lucas kanade feature tracker description of the algorithm[J]. Intel Corporation, 2001, 5 (10): 4.
15 KIM Deok-Hwa , KIM John-Hwan . Effective background model-based RGB-D dense visual odometry in a dynamic environment[J]. IEEE Transactions on Robotics, 2016, 32 (6): 1565- 1573.
doi: 10.1109/TRO.2016.2609395
16 ELGAMMAL Ahmed Mahmoud, HARWOOD David, DAVIS L. Non-parametric model for background subtraction[C]//European Conference on Computer Vision (ECCV). Berlin, Germany: Springer, 2000: 751-767.
17 KERL Christian, STURM Jurgen, CREMERS Daniel. Robust odometry estimation for RGB-D cameras[C]//International Conference on Robotics and Automation (ICRA). Karlsruhe, Germany: IEEE, 2013: 3748-3754.
18 XIAO Linhui , WANG Jinge , QIU Xiaoasong , et al. Dynamic-SLAM: semantic monocular visual localization and mapping based on deep learning in dynamic environment[J]. Robotics and Autonomous Systems, 2019, 117 (1): 1- 16.
19 LIU Wei, ANGUELOV Dragomir, ERHAN Dumitru, et al. SSD: single shot multibox detector[C]//European Conference on Computer Vision (ECCV). Amsterdam, Netherlands: Springer, 2016: 21-37.
20 JUDD Kevin M, GAMMELL Jonathan D, NEWMAN Paul. Multimotion visual odometry (mvo): simultaneous estimation of camera and third-party motions[C]//International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE, 2018: 3949-3956.
21 JUDD Kevin Michael , GAMMELL Jonathan D . The Oxford multimotion dataset: multiple SE (3) motions with ground truth[J]. IEEE Robotics and Automation Letters, 2019, 4 (2): 800- 807.
doi: 10.1109/LRA.2019.2892656
22 HUANG Jiahui, YANG Sheng, ZHAO Zishuo, et al. ClusterSLAM: a slam backend for simultaneous rigid body clustering and motion estimation[C]// International Conference on Computer Vision (ICCV). Seoul, Korea: IEEE, 2019: 5875-5884.
23 SOKAL Robert R , MICHENER Charles D . A statistical method for evaluating systematic relationships[J]. Univ Kansas (Sci Bull), 1958, 38 (1): 1409- 1438.
24 PRATIK Agarwal, GIAN Diego Tipaldi, LUCIANO Spinello, et al. Robust map optimization using dynamic covariance scaling[C]//International Conference on Robotics and Automation (ICRA). Karlsruhe, Germany: IEEE, 2013: 62-69.
25 SONG Shuran, YU Fisher, ZENG Andy, et al. Semantic scene completion from a single depth image[C]//Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii, USA: IEEE, 2017: 1746-1754.
26 DOSOVITSKIY Alexey, ROS German, CODEVILLA Felipe, et al. CARLA: an open urban driving simulator[C]// Proceeding of 1st Annual Conference on Robot Learning, Hamilton, New Zealand, 2017: 1-16.
27 RVNZ Martin, BUFFIER Maud, AGAPITO Lourdes, et al. Maskfusion: real-time recognition, tracking and reconstruction of multiple moving objects[C]//International Symposium on Mixed and Augmented Reality (ISMAR). München, Germany: IEEE, 2018: 10-20.
28 THOMAS Whelan, STEFAN Leutenegger, RENATO F Salas-Moreno, et al. ElasticFusion: dense SLAM without a pose graph[C]// Robotics: Science and Systems. Rome, Italy: DBIP, 2015: 1-9.
29 RVNZ Martin, AGAPITO Lourdes. Co-fusion: real-time segmentation, tracking and fusion of multiple objects[C]// International Conference on Robotics and Automation (ICRA). Marina Bay Sands, Singapore: IEEE, 2017: 4471-4478.
30 KELLER Maik, DAMIEN Lefloch, MARTIN Lambers, et al. Real-time 3d reconstruction in dynamic scenes using point-based fusion[C]// International Conference on 3D Vision-3DV (3DIMPVT). Washington, USA: IEEE, 2013: 1-8.
31 RALUCA Scona, MARIANO Jaimez, YVAN Petillot, et al. Staticfusion: background reconstruction for dense rgb-d slam in dynamic environments[C]// International Conference on Robotics and Automation (ICRA). Brisbane, Australia: IEEE, 2018: 1-9.
32 HUANG Jiahui, YANG Sheng, MU Taijiang, et al. ClusterVO: clustering moving instances and estimating visual odometry for self and surroundings[C]//Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020: 2168-2177.
33 REDMON Joseph, ALI Farhadi. YOLO9000: better, faster, stronger[C]//Computer Vision and Pattern Recognition (CVPR). Honolulu, Hawaii: IEEE, 2017: 2168-2177.
34 KRHENBüHL Philipp, KOLTUN Vladlen. Efficient inference in fully connected crfs with gaussian edge potentials[C]//Advances in Neural Information Processing Systems (NIPS). Granada, Spain: Curran Associate, 2011: 109-117.
35 LI Peiliang, QIN Tong, SHEN Shaojie. Stereo vision-based semantic 3d object and ego-motion tracking for autonomous driving[C]// European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 646-661.
36 HANDA Ankur, THOMAS Whelan, JOHN McDonald, et al. A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM[C]// International Conference on Robotics and Automation (ICRA). Miami, USA: IEEE, 2014: 1524-1531.
37 MOUSAVIAN Arsalan, ANGUELOV Dragomir, FLYNN John, et al. 3d bounding box estimation using deep learning and geometry[C]// Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 7074-7082.
38 REN Shaoqing, HE Kaiming, GIRSHICK Ross, et al. Faster r-cnn: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems (NIPS). Montreal, Canada: Curran Associate, 2015: 91-99.
39 LI Peiliang, CHEN Xiaozhi, SHEN Shaojie. Stereo R-CNN based 3D object detection for autonomous driving[C]//Computer Vision and Pattern Recognition (CVPR). California, USA: IEEE, 2019: 1902-09738.
40 BOLYA Daniel, ZHOU Chong, XIAO Fanyi, et al. YOLACT: real-time instance segmentation[C]// International Conference on Computer Vision (ICCV). Seoul, Korea: IEEE, 2019: 5875-5884.
41 CHEN Hao, SUN Kunyang, TIAN Zhi, et al. Blendmask: top-down meets bottom-up for instance segmentation[C]//Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020: 2168-2177.
42 Ultralytics. github[EB/OL]. (2020-4-1)[2020-8-13]. https://github.com/ultralytics/yolov5.
43 MIN Erxue , GUO Xifeng , LIU Qiang , et al. A survey of clustering with deep learning: from the perspective of network architecture[J]. IEEE Access, 2018, 6 (1): 39501- 39514.
44 BOWMAN Sean L, NIKOLAY Atanasov, KOSTAS Daniilidis, et al. Probabilistic data association for semantic slam[C]//International Conference on Robo-tics and Automation (ICRA). Marina Bay Sands, Singapore: IEEE, 2017: 1722-1729.
45 POHLEN Tobias, HERMANS Alexander, MATHIAS Markus, et al. Full-resolution residual networks for semantic segmentation in street scenes[C]//Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017: 4151-4160.
46 WU Dazhong , ROSEN David , WANG Lihui , et al. Cloud-based design and manufacturing: a new paradigm in digital manufacturing and design innovation[J]. Computer-Aided Design, 2015, 59 (1): 1- 14.
47 DU Zhihui, YANG Weiqiang, CHEN Yinong, et al. Design of a robot cloud center[C]// International Symposium on Autonomous Decentralized Systems. Hiroshima, Japan: IEEE, 2011: 269-275.
[1] 刘保成,朴燕,宋雪梅. 联合检测的自适应融合目标跟踪[J]. 山东大学学报 (工学版), 2020, 50(3): 51-57.
[2] 张胜男,王雷,常春红,郝本利. 基于三维剪切波变换和BM4D的图像去噪方法[J]. 山东大学学报 (工学版), 2020, 50(2): 83-90.
[3] 赵越男,陈桂友,孙琛,卢宁,廖立伟. 基于空间隐患分布与运动意图解析的危险评估方法[J]. 山东大学学报 (工学版), 2020, 50(1): 28-34.
[4] 姚元玺. 基于分场景重构的风电汇聚趋势性量化方法[J]. 山东大学学报 (工学版), 2019, 49(6): 86-92.
[5] 汪嘉晨,唐向红,陆见光. 轴承故障诊断中特征选取技术[J]. 山东大学学报 (工学版), 2019, 49(2): 80-87, 95.
[6] 刘世光,王海荣,刘锦. 快速四点一致性点云粗配准算法[J]. 山东大学学报 (工学版), 2019, 49(2): 1-7.
[7] 范君,业巧林,业宁. 基于改进的有监督无参局部保持投影算法的人脸识别[J]. 山东大学学报 (工学版), 2019, 49(1): 10-16.
[8] 王国新,陈凤东,刘国栋. 基于彩色伪随机编码结构光特征提取方法[J]. 山东大学学报 (工学版), 2018, 48(5): 55-60.
[9] 窦婷婷,姚元玺,陈鹏,芦灯. 基于ATP-EMTP的电弧建模及工程仿真[J]. 山东大学学报(工学版), 2018, 48(4): 102-108.
[10] 张宪红,张春蕊. 基于六维前馈神经网络模型的图像增强算法[J]. 山东大学学报(工学版), 2018, 48(4): 10-19.
[11] 丁筱玲,赵强,李贻斌,马昕. 基于模板匹配的改进型目标识别算法[J]. 山东大学学报(工学版), 2018, 48(2): 1-7.
[12] 赵晔,何潇,周东华. 带有不准确测量噪声的最小二乘故障估计[J]. 山东大学学报(工学版), 2017, 47(5): 254-262.
[13] 刘岩,李幼军,陈萌. 基于EMD和SVM的抑郁症静息态脑电信号分类研究[J]. 山东大学学报(工学版), 2017, 47(3): 21-26.
[14] 金培培,孙丰荣,刘芳蕾,姚桂华. 基于散斑跟踪技术的超声心动图心动周期估计[J]. 山东大学学报(工学版), 2017, 47(2): 94-99.
[15] 王斌,常发亮,刘春生. 基于多特征融合的交通标志分类[J]. 山东大学学报(工学版), 2016, 46(4): 34-40.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 王素玉,艾兴,赵军,李作丽,刘增文 . 高速立铣3Cr2Mo模具钢切削力建模及预测[J]. 山东大学学报(工学版), 2006, 36(1): 1 -5 .
[2] 张永花,王安玲,刘福平 . 低频非均匀电磁波在导电界面的反射相角[J]. 山东大学学报(工学版), 2006, 36(2): 22 -25 .
[3] 李 侃 . 嵌入式相贯线焊接控制系统开发与实现[J]. 山东大学学报(工学版), 2008, 38(4): 37 -41 .
[4] 施来顺,万忠义 . 新型甜菜碱型沥青乳化剂的合成与性能测试[J]. 山东大学学报(工学版), 2008, 38(4): 112 -115 .
[5] 孔祥臻,刘延俊,王勇,赵秀华 . 气动比例阀的死区补偿与仿真[J]. 山东大学学报(工学版), 2006, 36(1): 99 -102 .
[6] 来翔 . 用胞映射方法讨论一类MKdV方程[J]. 山东大学学报(工学版), 2006, 36(1): 87 -92 .
[7] 余嘉元1 , 田金亭1 , 朱强忠2 . 计算智能在心理学中的应用[J]. 山东大学学报(工学版), 2009, 39(1): 1 -5 .
[8] 李梁，罗奇鸣，陈恩红. 对象级搜索中基于图的对象排序模型（英文）[J]. 山东大学学报(工学版), 2009, 39(1): 15 -21 .
[9] 陈瑞,李红伟,田靖. 磁极数对径向磁轴承承载力的影响[J]. 山东大学学报(工学版), 2018, 48(2): 81 -85 .
[10] 王波,王宁生 . 机电装配体拆卸序列的自动生成及组合优化[J]. 山东大学学报(工学版), 2006, 36(2): 52 -57 .
版权所有 © 2018 《山东大学学报(工学版)》编辑部　
地址: 济南市山大南路27号（250100）　 电话: 0531-88366735　 E-mail: xbgxb@sdu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发