Journal of Shandong University(Engineering Science) ›› 2018, Vol. 48 ›› Issue (5): 69-76.doi: 10.6040/j.issn.1672-3961.0.2017.424

• Machine Learning & Data Mining • Previous Articles     Next Articles

Active driving behavior analysis algorithm based on monocular camera

Chenmou WU1(),Zhijun FANG1,*(),Jenqneng HWANG2   

  1. 1. School of Electrical and Electronic Engineering, Shanghai University of Engineering Science, Shanghai 201620, China
    2. Department of Electrical Engineering, University of Washington, Seattle 352500, Washington, USA
  • Received:2017-08-29 Online:2018-10-01 Published:2017-08-29
  • Contact: Zhijun FANG E-mail:747519579@qq.com;zjfang@foxmail.com
  • Supported by:
    国家自然科学基金资助项目(61461021);上海市科委地方院校能力建设资助项目(15590501300)

RichHTML

17

PDF (PC)

609

Abstract:

In order to prevent accidents, an algorithm for recognizing and monitoring the driver′s behavior based on the three-dimensional pose estimation of the human body was proposed. A monocular camera was used to capture the video stream of the driver in motion, the two-dimensional contour features of each frame of the image was extracted, and the two-dimensional projection of the pre-established three-dimensional human body model was matched to estimate the attitude of the driver′s upper body in real time. Based on the three-dimensional coordinates of the driver′s eight skeletal nodes, the driver′s behavior was identified and analyzed. Four driving states of driver′s normal, one-handed, answering calls and fatigue/drunk driving were simulated. Through the coordinate changes of the skeletal nodes, the gesture behavior of the driver could be detected and recognized, and the driver could be given reminders. When the light was enough, the algorithm could reduce the false detection rate by 24.24% compared with the PRECLOSE algorithm.

Key words: traffic accident, driving condition, monocular camera, pose estimation, behavior recognition, simulate anneal

CLC Number: 

  • TP391

Fig.1

The flow chart of algorithm"

Fig.2

Example of edge"

Fig.3

Example of skin color"

Fig.4

Example of silhouette"

Fig.5

The flow chart of simulated annealing algorithm"

Fig.6

Four cases of driver behavior"

Fig.7

Example of upper body pose estimation with long sleeve"

Fig.8

Example of upper body pose estimation with short sleeve"

Fig.9

Four cases of driving behavior pose estimation"

Table 1

3D coordinates of normal driving"

部位 x y z
头 7.302 7.639 8.699
右肩 5.985 8.113 5.019
左肩 7.258 6.809 1.800
躯干 16.369 17.921 11.958
右肘 29.141 31.112 9.220
左肘 24.823 22.337 5.668
右手 34.776 28.653 14.690
左手 27.992 36.542 14.340

Table 2

3D coordinates of driving with one hand"

部位 x y z
头 4.998 5.229 5.954
右肩 4.097 5.553 3.436
左肩 4.968 4.661 1.232
躯干 11.205 12.267 8.185
右肘 19.947 21.296 6.311
左肘 26.669 33.323 3.961
右手 19.161 25.013 9.816
左手 23.804 19.613 10.055

Table 3

3D coordinates of answering calls"

部位 x y z
头 4.302 6.639 6.699
右肩 4.985 5.113 3.019
左肩 4.258 4.809 1.821
躯干 11.369 12.921 8.958
右肘 20.141 31.112 6.922
左肘 16.991 15.290 3.880
右手 7.776 11.653 4.690
左手 20.112 21.39 7.816

Table 4

3D coordinates of drunk/fatigue driving"

部位 x y z
头 1.084 1.079 0.473
右肩 0.980 1.510 0.467
左肩 0.970 2.245 0.063
躯干 5.914 6.107 3.806
右肘 7.958 5.841 8.434
左肘 7.673 7.564 8.418
右手 9.266 11.352 7.505
左手 8.420 10.527 6.721

Fig.10

Ratio of the distance between the shoulder and hands"

Fig.11

Distance of two hands and hand to head"

Fig.12

Curve of head in x-axis"

Fig.13

Curve of head in y-axis"

Fig.14

Curve of head in z-axis"

Table 5

Comparison bewteen our algorithm and PERCLOS"

算法 模拟疲劳驾驶次数 检测疲劳驾驶次数 误检次数
本研究算法 12 13 1
PERCLOS 12 16 4

Table 6

Results of algorithmic in different lighting environments"

性能指标 总帧数 真实疲劳驾驶次数 检测疲劳驾驶次数 真实打电话次数 检测打电话次数 真实单手次数 检测单手次数
光线较好 15 680 3 3 3 3 3 2
光线较差 10 079 3 2 3 1 3 4
1 国家统计局.中华人民共和国2016年国民经济和社会发展统计公报[R/OL].[2017-02-28]. http://www.stats.gov.cn/tjsj/zxfb/201702/t20170228_1467424.html.
2 NOBE S A, WANG F Y. An overview of recent developments in automated lateral and longitudinal vehicle controls[C]//Proceedings of IEEE International Conference on Systems, Man, and Cybernetics. Tucson, USA: IEEE, 2002: 3447-3452.
3 TAKEI Y, FURUKAWA Y. Estimate of driver's fatigue through steering motion[C]//Proceedings of IEEE International Conference on Systems, Man and Cybernetics. Waikoloa, USA: IEEE, 2006: 1765-1770.
4 DAS D , ZHOU S , LEE J D . Differentiating alcohol-Induced driving behavior using steering wheel signals[J]. IEEE Transactions on Intelligent Transportation Systems, 2012, 13 (3): 1355- 1368.
doi: 10.1109/TITS.2012.2188891
5 SIMONS R , MARTENS M , RAMAEKERS J , et al. Effects of dexamphetamine with and without alcohol on simulated driving[J]. Psychopharmacology, 2012, 222 (3): 391- 399.
doi: 10.1007/s00213-011-2549-0
6 METS M A , KUIPERS E , LM D S D , et al. Effects of alcohol on highway driving in the STISIM driving simulator[J]. Human Psychopharmacology Clinical & Experimental, 2011, 26 (6): 434- 439.
7 SAHAYADHAS A , SUNDARAJ K , MURUGAPPAN M . Detecting driver drowsiness based on sensors: a review[J]. Sensors, 2012, 12 (12): 16937- 16953.
doi: 10.3390/s121216937
8 INGRE M , AKERSTEDT T , PETERS B , et al. Subjective sleepiness, simulated driving performance and blink duration: examining individual differences[J]. Journal of Sleep Research, 2010, 15 (1): 47- 53.
9 KE S R, ZHU L J, HWANG J N, et al. Real-time 3D human pose estimation from monocular view with applications to event detection and video gaming[C]//Proceedings of 2010 Seventh IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). Boston, USA: IEEE, 2010: 489-496.
10 SUN Ruizhi.3D front-view human upper body pose estimation using single camera[D]. Seattle: University of Washington, 2013.
11 FELZENSZWALB P , MCALLESTER D , RAMANAN D . A discriminatively trained, multiscale, deformable part model[J]. Cvpr, 2008, (8): 1- 8.
12 HAN F. A two-stage approach to people and vehicle detection with HOG-based SVM[C]//Proceedings of Proc. of Workshop on Performance Metrics for Intelligent Systems. Piscataway, NJ: IEEE, 2006: 133-140.
13 VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Kauai, Hawaii: IEEE, 2001: 511.
14 LIENHART R, MAYDT J. An extended set of Haar-like features for rapid object detection[C]//Proceedings of International Conference on Image Processing, Rochester. NY, USA: IEEE, 2002.
15 LI H, LIN Z, SHEN X, et al. A convolutional neural network cascade for face detection[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015). Boston, MA, USA: IEEE, 2015: 5325-5334.
16 CHIEN Y . Pattern classification and scene analysis[J]. The Library Quarterly: Information, Community, Policy, 1974, 44 (3): 462- 463.
17 YOON I , KIM S , KIM D , et al. Adaptive defogging with color correction in the HSV color space for consumer surveillance system[J]. IEEE Transactions on Consumer Electronics, 2012, 58 (1): 606- 607.
18 SCHWARZ M W , COWAN W B , BEATTY J C . An experimental comparison of RGB, YIQ, LAB, HSV, and opponent color models[J]. Acm Transactions on Graphics, 1987, 6 (2): 123- 158.
doi: 10.1145/31336.31338
19 VECCHI M P , KIRKPATRICK S . Global wiring by simulated annealing[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2004, 2 (4): 215- 222.
20 TRUTSCHEL U, SIROIS B, SOMMER D, et al. PERCLOS: An alertness measure of the past[C]//Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design. California, USA: IEEE, 2011: 172-179.
[1] SHI Qingxuan, WANG Qian, TIAN Xuedong. Cascaded tracklet-based spatio-temporal model for video pose estimation [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 14-21.
[2] WEI Wei, ZHANG Yanning. Pose estimation based on semi-supervised latent Dirichlet allocation [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(3): 17-22.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] MENG Jian, LI Yibin, LI Bin. Bound gait controlling method of quadruped robot[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(3): 28 -34 .
[2] HE Dongzhi, ZHANG Jifeng, ZHAO Pengfei. Parallel implementing probabilistic spreading algorithm using MapReduce programming mode[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 22 -28 .
[3] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 77 -81 .
[4] WANG Huan, ZHOU Zhongmei. An over sampling algorithm based on clustering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 134 -139 .
[5] XIAO Qiao, PEI Jihong, WANG Lixia, GONG Zhicheng. Ship detection in remote sensing image based on the fuzzy fusion of multi-channel Gabor filtering[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 29 -35 .
[6] MA Xiangming, SUN Xia, ZHANG Qiang. Construction and analysis on typical working cycle of wheel loader[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 82 -87 .
[7] WANG Lanzhong, MENG Wenjie. Video perceptual encryption algorithm in remote education receiver[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(4): 40 -44 .
[8] LIANG Zehua, CUI Yaodong, ZHANG Yu. The one-dimensional cutting stock problem with sequence-dependent cut losses[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 75 -80 .
[9] ZHU Dengyuan, YAO Zhanyong, GE Shouren. Study on buried abutment slope transformation based on numerical simulation and site detections[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(3): 86 -94 .
[10] YE Mingquan, GAO Lingyun, WAN Chunyuan. Gene expression data classification based on artificial bee colony and SVM[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 10 -16 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd