Journal of Shandong University(Engineering Science) ›› 2019, Vol. 49 ›› Issue (5): 98-104.doi: 10.6040/j.issn.1672-3961.0.2018.348

• Machine Learning & Data Mining • Previous Articles     Next Articles

Object detection of 3D point clouds based on F-PointNet

Peng WAN()   

  1. School of Computer Science and Engineering, Nanjing University of Science and Technology, NanJing 210094, Jiangsu, China
  • Received:2018-08-14 Online:2019-10-20 Published:2019-10-18

RichHTML

55

PDF (PC)

1318

Abstract:

Aiming at the problem of poor detection accuracy of the current 3D point cloud object detection model, the F-PointNet model, which directly processed point cloud data, was used to detect cars, pedestrians and cyclists, and the model was fine-tuned to further improve the object detection accuracy. The model was tested by different parameter initialization methods, $\ell $2 regularization and modifying convolution kernels. The experimental results showed that the Xavier parameter initialization method converged faster 0.09s than the truncated normal distribution method, and the vehicle detection accuracy and the cyclists detection accuracy was about 3% and 2% higher respectively. By adding $\ell $2 regularization, the detection accuracy of pedestrians and cyclists was increased by about 2% and 1% respectively. By reducing the number of convolution kernels in the first layer of T-Net (Transformer Networks) to 128, the detection accuracy of cars and cyclists was increased by about 1% and 2% respectively, which confirmed that the model could effectively improve object detection accuracy.

Key words: deep learning, 3D point cloud, object detection, detection accuracy, F-PointNet model

CLC Number: 

  • TP249

Fig.1

Structure of F-PointNet model"

Fig.2

Coordinate for point cloud"

Fig.3

Structure of 3D instance segmentation PointNet model"

Fig.4

Structure of the T-Net model"

Fig.5

Structure of the amodal box estimation PointNet model"

Fig.6

Structure of the changed T-Net model"

Table 1

Object accuracy results under different initialization methods"

%
初始化方法 运行时间/s 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
Xavier 0.13 85.18 71.73 63.85 65.77 55.49 49.66 70.62 52.97 49.97
截断正态 0.22 82.24 68.84 60.86 67.29 56.54 50.00 68.03 50.24 46.91

Table 2

Object detection accuracy at different decay rates"

%
正则化系数 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
0.000 5 84.66 70.69 63.37 69.08 59.09 51.68 67.94 51.24 47.33
0.001 0 83.47 69.39 62.81 66.34 55.85 49.18 68.52 51.47 47.96
0.010 0 85.53 70.89 62.81 67.38 58.05 50.61 71.18 53.62 50.12
0 85.71 71.73 63.85 65.77 55.49 49.66 70.62 52.97 49.97

Fig.7

Curve of learning rate with the batch processing times"

Fig.8

Curve of total loss with the batch processing times"

Table 3

Comparison of the accuracy for different models"

%
模型 汽车 行人 骑车人
简单 中等 困难 简单 中等 困难 简单 中等 困难
MV3D 71.29 62.68 56.56
AVOD 84.41 74.44 68.65
VoxelNet 81.97 65.46 62.85 57.86 53.42 48.87 67.17 47.65 45.11
F-PointNet 84.73 70.56 62.37 67.26 57.37 50.28 68.61 50.97 47.40
Ours 85.71 71.73 63.85 66.77 56.61 49.66 70.62 52.97 49.97

Fig.9

Visualization comparison of images, original point clouds, and point clouds with bounding boxes"

1 薛瑞.基于RGB-D数据的点云配准[D].西安:长安大学, 2017.
XUE Rui. Point cloud registration based on RGB-D data[D]. Xi'an: Chang'an University, 2017.
2 赵熙.基于地面激光扫描面点云数据的三维重建方法研究[D].武汉:武汉大学, 2010.
ZHAO Xi. Research on 3D reconstruction method based on surface laser scanning point cloud data[D]. Wuhan: Wuhan University, 2010.
3 MATURAN D, SCHERER S. VoxNet: a 3D convolutional neural network for real-time object recognition[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE Press, 2015: 922-928.
4 WU Z, SONG S, KHOSLA A, et al. 3d shapenets: a deep representation for volumetric shapes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE Press, 2015: 1912-1920.
5 LI B. 3D fully convolutional network for vehicle detection in point cloud[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, Canada: IEEE Press, 2017: 1513-1518.
6 WANG D Z, POSNER I, WANG D Z, et al. Voting for voting in online point cloud object detection[C]//Robotics: Science and Systems. Rome, Italy: IEEE Press, 2015: 1317-1325.
7 ENGELCKE M , RAO D , WANG D Z , et al. Vote3Deep: fast object detection in 3D point clouds using efficient convolutional neural networks[J]. ICRA, 2016, 1609, 1355- 1361.
8 LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39 (4): 640- 651.
9 QI C R, SU H, NIWBNER M, et al. Volumetric and multi-view cnns for object classification on 3d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Press, 2016: 5648-5656.
10 SU H, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3d shape recognition[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile: IEEE Press, 2015: 945-953.
11 LI B, ZHANG T, XIA T. Vehicle detection from 3D lidar using fully convolutional network[C]//Robotics: Science and System. Ann Arbor, USA: IEEE Press, 2016: 1608-1616.
12 CHEN X , MA H , WAN J , et al. Multi-view 3D object detection network for autonomous driving[J]. Computer Vision and Pattern Recognition(CVPR), 2016, (10): 6526- 6534.
13 GONZALEZ A , VAZQUEZ D , LOPEZ A M , et al. On-board object detection: Multicue, multimodal, and multiview random forest of local experts[J]. IEEE Transactions on Cybernetics, 2017, 47 (11): 3980- 3990.
doi: 10.1109/TCYB.2016.2593940
14 ENZWEILER M , GAVRILA D M . A multilevel mixture-of-experts framework for pedestrian classification[J]. Image Processing IEEE Transactions, 2011, 20 (10): 2967- 2979.
doi: 10.1109/TIP.2011.2142006
15 QI C R, LIU W, WU C, et al. Frustum pointnets for 3d object detection from rgb-d data[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 918-927.
16 CHARLES R Q, SU H, MO K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Press, 2017: 652-660.
17 GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving: the KITTI vision benchmark suite[C]//IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE Computer Society, 2012: 3354-3361.
18 GEIGER A , LENZ P , STILLER C , et al. Vision meets robotics: the KITTI dataset[J]. International Journal of Robotics Research, 2013, 32 (11): 1231- 1237.
doi: 10.1177/0278364913491297
19 ZHOU Y, TUZEL O. Voxelnet: end-to-end learning for point cloud based 3d object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Press, 2018: 4490-4499.
20 KU J, MOZIFIAN M, LEE J, et al. Joint 3d proposal generation and object detection from view aggregation[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid, Spain: IEEE Press, 2018: 1-8.
[1] LI Changgang, LI Baoliang, CAO Yongji, WANG Jiaying. Review and prospect on artificial intelligence application in power system power flow calculation [J]. Journal of Shandong University(Engineering Science), 2025, 55(5): 1-17.
[2] SUO Daxiang, LI Bo. Photovoltaic defect detection based on fine-grained feature enhancement and scale matching [J]. Journal of Shandong University(Engineering Science), 2025, 55(4): 9-17.
[3] ZHOU Qunying, SUI Jiacheng, ZHANG Ji, WANG Hongyuan. Industrial product surface defect detection based on self supervised convolution and parameter free attention mechanism [J]. Journal of Shandong University(Engineering Science), 2025, 55(4): 40-47.
[4] XUE Bingbing, WANG Yong, YANG Weihao, WANG Chuan, YU Di, WANG Xu. Real-time expressway traffic data imputation and state prediction based on ETC system data [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 58-71.
[5] DONG Mingshu, CHEN Liqi, MA Chuanyi, ZHANG Zhuhao, SUN Renjuan, GUAN Yanhua, ZHUANG Peizhi. Deep learning-based intelligent judgment for radar detection of pavement cracks [J]. Journal of Shandong University(Engineering Science), 2025, 55(3): 72-79.
[6] Jiachun LI,Bowen LI,Jianbo CHANG. An efficient and lightweight RGB frame-level face anti-spoofing model [J]. Journal of Shandong University(Engineering Science), 2023, 53(6): 1-7.
[7] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
[8] Tongyu JIANG, Fan CHEN, Hongjie HE. Lightweight face super-resolution network based on asymmetric U-pyramid reconstruction [J]. Journal of Shandong University(Engineering Science), 2022, 52(1): 1-8.
[9] Jianqing WU,Xiuguang SONG. Review on development of simultaneous localization and mapping technology [J]. Journal of Shandong University(Engineering Science), 2021, 51(5): 16-31.
[10] YANG Xiuyuan, PENG Tao, YANG Liang, LIN Hongfei. Adaptive multi-domain sentiment analysis based on knowledge distillation [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 15-21.
[11] Qingfa CHAI,Shoujing SUN,Jifu QIU,Ming CHEN,Zhen WEI,Wei CONG. Prediction method of power grid emergency supplies under meteorological disasters [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 76-83.
[12] LIAO Jinping, MO Yuchang, YAN Ke. Model and application of short-term electricity consumption forecast based on C-LSTM [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 90-97.
[13] Guoyong CAI,Xinhao HE,Yangyang CHU. Visual sentiment analysis based on spatial attention mechanism and convolutional neural network [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 8-13.
[14] LIU Shuai, WANG Lei, DING Xutao. Emotional EEG recognition based on Bi-LSTM [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 35-39.
[15] Chunyang LI,Nan LI,Tao FENG,Zhuhe WANG,Jingkai MA. Abnormal sound detection of washing machines based on deep learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 108-117.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] YUE Yuan-Zheng. Relaxation in glasses far from equilibrium[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 1 -20 .
[2] LI Hui-ping, ZHAO Guo-qun, ZHANG Lei, HE Lian-fang. The development status of hot stamping and quenching of ultra high-strength steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 69 -74 .
[3] LI Shijin, WANG Shengte, HUANG Leping. Change detection with remote sensing images based on forward-backward heterogenicity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 1 -9 .
[4] LI Shu-cai,WANG Zhao-qing,LI Shu-chen . A polygonal finite element method based on irrational function interpolation[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 66 -70 .
[5] RUAN Jing,JI Lin,ZHU Ping-peng . Structure style selection of the mid-tower of a three-tower suspension bridge[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 106 -111 .
[6] XUE Yi-Bing, MENG Guang, ZHANG Le. A comparison study of two types of solar air collector performance[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(6): 147 -149 .
[7] MOU Xin-wei,XIE Shao-bin,JU Zhan-sheng . Link properties and electromagnetism calculation inairground HF communication[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(6): 71 -73 .
[8] QIU Dao-hong1, ZHANG Le-wen1, CUI Wei2, SU Mao-xin1, SUN Huai-feng1. A genetic neural network model based on a trend examination method and engineering application[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 113 -118 .
[9] WANG Juan,CHEN Hui-yan,DING Hua-rong . Start process control of a hydrodynamic-mechanical automatic transmission[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(2): 23 -27 .
[10] LU Dong,LI Jian-feng,SUN Jie,JIANG Feng . Finite element analysis for the verification of the dynamic clamping force during machining of the aero frame shape workpiece[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(1): 19 -22 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd