Journal of Shandong University(Engineering Science) ›› 2019, Vol. 49 ›› Issue (5): 98-104.doi: 10.6040/j.issn.1672-3961.0.2018.348

• Machine Learning & Data Mining • Previous Articles     Next Articles

Object detection of 3D point clouds based on F-PointNet

Peng WAN()   

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, Jiangsu, China
  • Received:2018-08-14 Online:2019-10-20 Published:2019-10-18

RichHTML

51

PDF (PC)

1002

Abstract:

Aiming at the problem of poor detection accuracy of the current 3D point cloud object detection model, the F-PointNet model, which directly processed point cloud data, was used to detect cars, pedestrians and cyclists, and the model was fine-tuned to further improve the object detection accuracy. The model was tested by different parameter initialization methods, $\ell $2 regularization and modifying convolution kernels. The experimental results showed that the Xavier parameter initialization method converged faster 0.09s than the truncated normal distribution method, and the vehicle detection accuracy and the cyclists detection accuracy was about 3% and 2% higher respectively. By adding $\ell $2 regularization, the detection accuracy of pedestrians and cyclists was increased by about 2% and 1% respectively. By reducing the number of convolution kernels in the first layer of T-Net (Transformer Networks) to 128, the detection accuracy of cars and cyclists was increased by about 1% and 2% respectively, which confirmed that the model could effectively improve object detection accuracy.

Key words: deep learning, 3D point cloud, object detection, detection accuracy, F-PointNet model

CLC Number: 

  • TP249

Fig.1

Structure of F-PointNet model"

Fig.2

Coordinate for point cloud"

Fig.3

Structure of 3D instance segmentation PointNet model"

Fig.4

Structure of the T-Net model"

Fig.5

Structure of the amodal box estimation PointNet model"

Fig.6

Structure of the changed T-Net model"

Table 1

Object accuracy results under different initialization methods"

%
初始化方法 运行时间/s 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
Xavier 0.13 85.18 71.73 63.85 65.77 55.49 49.66 70.62 52.97 49.97
截断正态 0.22 82.24 68.84 60.86 67.29 56.54 50.00 68.03 50.24 46.91

Table 2

Object detection accuracy at different decay rates"

%
正则化系数 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
0.000 5 84.66 70.69 63.37 69.08 59.09 51.68 67.94 51.24 47.33
0.001 0 83.47 69.39 62.81 66.34 55.85 49.18 68.52 51.47 47.96
0.010 0 85.53 70.89 62.81 67.38 58.05 50.61 71.18 53.62 50.12
0 85.71 71.73 63.85 65.77 55.49 49.66 70.62 52.97 49.97

Fig.7

Curve of learning rate with the batch processing times"

Fig.8

Curve of total loss with the batch processing times"

Table 3

Comparison of the accuracy for different models"

%
模型 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
MV3D 71.29 62.68 56.56
AVOD 84.41 74.44 68.65
VoxelNet 81.97 65.46 62.85 57.86 53.42 48.87 67.17 47.65 45.11
F-PointNet 84.73 70.56 62.37 67.26 57.37 50.28 68.61 50.97 47.40
Ours 85.71 71.73 63.85 66.77 56.61 49.66 70.62 52.97 49.97

Fig.9

Visualization comparison of images, original point clouds, and point clouds with bounding boxes"

1 薛瑞.基于RGB-D数据的点云配准[D].西安:长安大学, 2017.
XUE Rui. Point cloud registration based on RGB-D data[D]. Xi'an: Chang'an University, 2017.
2 赵熙.基于地面激光扫描面点云数据的三维重建方法研究[D].武汉:武汉大学, 2010.
ZHAO Xi. Research on 3D reconstruction method based on surface laser scanning point cloud data[D]. Wuhan: Wuhan University, 2010.
3 MATURAN D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE Press, 2015: 922-928.
4 WU Z, SONG S, KHOSLA A, et al. 3d shapenets: a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE Press, 2015: 1912-1920.
5 LI B. 3D fully convolutional network for vehicle detection in point cloud[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, Canada: IEEE Press, 2017: 1513-1518.
6 WANG D Z, POSNER I, WANG D Z, et al. Voting for voting in online point cloud object detection[C]//Robotics: Science and Systems. Rome, Italy: IEEE Press, 2015: 1317-1325.
7 ENGELCKE M , RAO D , WANG D Z , et al. Vote3Deep: fast object detection in 3D point clouds using efficient convolutional neural networks[J]. ICRA, 2016, 1609, 1355- 1361.
8 LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39 (4): 640- 651.
9 QI C R, SU H, NIWBNER M, et al. Volumetric and multi-view cnns for object classification on 3d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 5648-5656.
10 SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3d shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Press, 2015: 945-953.
11 LI B, ZHANG T, XIA T. Vehicle detection from 3D lidar using fully convolutional network[C]//Robotics: Science and System. Ann Arbor, USA: IEEE Press, 2016: 1608-1616.
12 CHEN X , MA H , WAN J , et al. Multi-view 3D object detection network for autonomous driving[J]. Computer Vision and Pattern Recognition(CVPR), 2016, (10): 6526- 6534.
13 GONZALEZ A , VAZQUEZ D , LOPEZ A M , et al. On-board object detection: Multicue, multimodal, and multiview random forest of local experts[J]. IEEE Transactions on Cybernetics, 2017, 47 (11): 3980- 3990.
doi: 10.1109/TCYB.2016.2593940
14 ENZWEILER M , GAVRILA D M . A multilevel mixture-of-experts framework for pedestrian classification[J]. Image Processing IEEE Transactions, 2011, 20 (10): 2967- 2979.
doi: 10.1109/TIP.2011.2142006
15 QI C R, LIU W, WU C, et al. Frustum pointnets for 3d object detection from rgb-d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 918-927.
16 CHARLES R Q, SU H, MO K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Press, 2017: 652-660.
17 GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving: the KITTI vision benchmark suite[C]//IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE Computer Society, 2012: 3354-3361.
18 GEIGER A , LENZ P , STILLER C , et al. Vision meets robotics: the KITTI dataset[J]. International Journal of Robotics Research, 2013, 32 (11): 1231- 1237.
doi: 10.1177/0278364913491297
19 ZHOU Y, TUZEL O. Voxelnet: end-to-end learning for point cloud based 3d object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 4490-4499.
20 KU J, MOZIFIAN M, LEE J, et al. Joint 3d proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE Press, 2018: 1-8.
[1] Ji ZHANG,Cui JIN,Hongyuan WANG,Shoubing CHEN. Pedestrian recognition based on singular value decomposition pedestrian alignment network [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 91-97.
[2] Zhixiang LIANG,Xiaoming LIU,Ying MU,Yutian LIU. Prediction method of wind power and PV ramp event based on deep learning [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 24-28.
[3] Yutian LIU,Runjia SUN,Hongtao WANG,Xueping GU. Review on application of artificial intelligence in power system restoration [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 1-8.
[4] Lizhao LI,Guoyong CAI,Jiao PAN. A microblog rumor events detection method based on C-GRU [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 102-106, 115.
[5] Xiaoxiong HOU,Xinzheng XU,Jiong ZHU,Yanyan GUO. Computer aided diagnosis method for breast cancer based on AlexNet and ensemble classifiers [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 74-79.
[6] Chengbin ZHANG,Hui ZHAO,Zongyu CAO. The vulnerability mining method for KWP2000 protocol based on deep learning and fuzzing [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 17-22.
[7] XIE Zhifeng, WU Jiaping, MA Lizhuang. Chinese financial news classification method based on convolutional neural network [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 34-39.
[8] TANG Leshuang, TIAN Guohui, HUANG Bin. An object fusion recognition algorithm based on DSmT [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(1): 50-56.
[9] ZHOU Funa, GAO Yulin, WANG Jiayu, WEN Chenglin. Early diagnosis and life prognosis for slowlyvarying fault based on deep learning [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(5): 30-37.
[10] HUI Kaifa, CHENG Keyang, ZHAN Yongzhao. The video synopsis based on the enhanced ViBe algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 43-48.
[11] LIU Yingxia, WANG Xichang, TANG Xiaoli, CHANG Faliang. Object detection algorithm based on Bayesian probability estimation in wavelet domain [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(2): 63-70.
[12] HE Zhengyi, ZENG Xianhua, QU Shengwei, WU Zhilong. The time series prediction model based on integrated deep learning [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(6): 40-47.
[13] ZHENG Yi, ZHU Chengzhang. A prediction method of atmospheric PM2.5 based on DBNs [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(6): 19-25.
[14] QIAO Wei1, WANG Hui-yuan1,2, WU Xiao-juan1, LIU Peng-wei1. Crowd object detection and classification based on a chaotic dynamic model [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(2): 19-23.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[2] LI Ke,LIU Chang-chun,LI Tong-lei . Medical registration approach using improved maximization of mutual information[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 107 -110 .
[3] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[4] XU Li-li,JI Zhong,XIA Ji-mei . The optimum algorithm for the container loading problem with homogeneous cargoes[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 14 -17 .
[5] CHEN Huaxin, CHEN Shuanfa, WANG Binggang. The aging behavior and mechanism of base asphalts[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 125 -130 .
[6] XU Xiaodan, DUAN Zhengjie, CHEN Zhongyu. The sentiment mining method based on extended sentiment dictionary and integrated features[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(6): 15 -18 .
[7] WANG Xue-ping,WANG Deng-jie,SUN Ying-ming*,DONG Lei . Application of the nonprism total station in the detection of a highway bridge[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(3): 105 -108 .
[8] ZHANG Bo,LI Shu-cai,YANG Xue-ying,WANG Xi-ping,ZHANG Dun-fu . Numerical analysis on the stability of a rocksalt roadbed with two circular cavities [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 66 -69 .
[9] LI Shu-cai,WANG Zhao-qing,LI Shu-chen . A polygonal finite element method based on irrational function interpolation[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 66 -70 .
[10] CAO Gang, DONG Chao-Yang, HUANG Ji-Bao, XUE Yu-Qing. Power system inter-area oscillation damping control with FACTS devies[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 31 -36 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd