Journal of Shandong University(Engineering Science) ›› 2024, Vol. 54 ›› Issue (2): 1-12.doi: 10.6040/j.issn.1672-3961.0.2023.279

• Transportation Engineering—Special Issue for Intelligent Transportation •     Next Articles

A research survey of driver drowsiness driving detection

YANG Jucheng1*, WEI Feng2,1, LIN Liang1, JIA Qingxiang1, LIU Jianzheng1   

  1. 1. College of Artificial Intelligence, Tianjin University of Science and Technology, Tianjin 300457, China;
    2. College of Mechanical Engineer, Tianjin University of Science and Technology, Tianjin 300457, China
  • Online:2024-04-20 Published:2024-04-17

CLC Number: 

  • TP391
[1] 李都厚,刘群,袁伟,等. 疲劳驾驶与交通事故关系[J]. 交通运输工程学报, 2010, 10(2): 104-109. LI Duhou, LIU Qun, YUAN Wei, et al. Relationship betweenfatigue driving and traffic accidents[J]. Journal of Traffic and Transportation Engineering, 2010, 10(2): 104-109.
[2] National Highway Traffic Safety Adminstration. Asleep at the wheel-compendium synopsis[EB/OL].(2015-11-04)[2023-11-10]. https://www.nhtsa.gov/drowsy-driving/asleep-wheel-compendium-synopsis.
[3] KHUMPISUTH O, CHOTCHINASRI T, KOSHAKOSAI V, et al. Driver drowsiness detection using eye-closeness detection[C] //Processdings of 2016 12th International Conference on Signal-Image Technology and Internet-Based Systems(SITIS). Piscataway, USA: IEEE, 2016:661-668.
[4] BROWN I D. Driver fatigue[J]. Human Factors, 1994, 36(2): 298-314.
[5] 张旭欣,王雪松. 疲劳驾驶研究与预防最新进展[J]. 汽车与安全, 2019, 4. ZHANG Xuxin, WANG Xuesong. Latest developments in fatigue driving research and prevention[J]. Auto and Safety, 2019, 4.
[6] 孟宪超. 疲劳驾驶交通事故特点及有效预防分析[J]. 物流工程与管理, 2014, 36(8): 187-188. MENG Xianchao. Driving fatigue caused by traffic accident characteristics and effective prevention analysis[J]. Preservation of Commodities, 2014, 36(8): 187-188.
[7] CORTACERO K, FISHER T, DEMIRIS Y. RT-BENE: A dataset and baselines for real-time blink estimation in natural environments[C] //Processdings of the IEEE/CVF International Conference on Computer Vision Workshops. Piscataway, USA: IEEE, 2019: 339-357.
[8] PAN G, SUN L, WU Z, et al. Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C] //Processdings of 2007 IEEE 11th International Conference on Computer Vision. Piscataway, USA: IEEE, 2007: 1-8.
[9] SONG F, TAN X, LIU X, et al. Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients[J]. Pattern Recognition, 2014, 47(9): 2825-2838.
[10] HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild:a database for studying face recognition in unconstrained environments[C] //Processdings of Workshop on Faces in Real-Life Images: Detection, Alignment, and Recognition. California, USA: Hans Publishers, 2008: 1-15.
[11] KASSEM H A, CHOWDHURY M, ABAWAJY J, et al. Yawn based driver fatigue level prediction[J]. EPiC Series in Computing, 2020, 10(69): 372-382.
[12] WENG C H, LAI Y H, LAI S H. Driver drowsiness detection via a hierarchical temporal deep belief network[C] //Processdings of Computer Vision-ACCV 2016 Workshops. Berlin, German: Springer, 2017: 117-133.
[13] GHODDOOSIAN R, GALIB M, ATHITSOS V. A realistic dataset and baseline temporal model for early drowsiness detection[C] //Processdings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, USA: IEEE, 2019: 178-187.
[14] MASSOZ Q, LANGOHR T, FRANCOIS C, et al. The ULg multimodality drowsiness database(called DROZY)and examples of use[C] //Processdings of 2016 IEEE Winter Conference on Applications of Computer Vision(WACV). Piscataway, USA: IEEE, 2016: 1-7.
[15] ROTH M, GAVRILA D M. DD-pose-a large-scale driver head pose benchmark[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 927-934.
[16] DIAZ C K, HERN A, LOPEZ A M. A reduced feature set for driver head pose estimation[J]. Applied Soft Computing, 2016, 45: 98-107.
[17] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015:3659- 3667.
[18] BORGHI G, VENTURELLI M, VEZZANI R, et al. Poseidon: face-from-depth for driver pose estimation[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2017: 4661-4670.
[19] MARTIN M, ROITBERG A, HAURILET M, et al. Drive and act: a multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles[C] //Processdings of the IEEE/CVF International Conference on Computer Vision. Piscataway, USA: IEEE, 2019: 2801-2810.
[20] ORTEGA J D, KOSE N, CANAS P, et al. Dmd: a large-scale multi-modal driver monitoring dataset for attention and alertness analysis[C] //Processdings of Computer Vision-ECCV 2020 Workshops. Berlin, German: Springer, 2020: 387-405.
[21] YANG C, YANG Z, LI W, et al. FatigueView: amulti-camera video dataset for vision-based drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(1): 233-246.
[22] LI Y, WEI J, LIU Y, et al. Deep learning for micro-expression recognition: a survey[J]. IEEE Transactions on Affective Computing, 2022, 13(4): 2028-2046.
[23] VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C] //Processdings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2001, 1: 511-518.
[24] RAHMAN A, SIRSHAR M, KHAN A. Real time drowsiness detection using eye blink monitoring[C] //Processdings of 2015 National Software Engineering Conference(NSEC). Piscataway,USA: IEEE, 2015: 1-7.
[25] SOMMER D, GOLZ M. Evaluation of PERCLOS based current fatigue monitoring technologies[C] //Processdings of 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. Piscataway,USA: IEEE, 2010: 4456-4459.
[26] BACCOUR M H, DRIEWER F, KASNEC E, et al. Camera-based eye blink detection algorithm for assessing driver drowsiness[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 987-993.
[27] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015: 3659-3667.
[28] YAN J J, KUO H H, LIN Y F, et al. Real-time driver drowsiness detection system based on PERCLOS and grayscale image processing[C] //Processdings of 2016 International Symposium on Computer, Consumer and Control(IS3C). Piscataway,USA: IEEE, 2016: 243-246.
[29] 江跃龙,张铭智. 基于PERCLOS的列车司机疲劳检测设计与实现[J]. 计算机时代, 2023(4): 112-115. JIANG Yuelong, ZHANG Mingzhi. Design and implementation of train driver fatigue detection based on PERCLOS[J]. Computer Age, 2023(4): 112-115.
[30] KUWAHARA A, HIRAKAWA R, KAWANO H, et al. Eye fatigue prediction system using blink detection based on eye image[C] //Processdings of 2021 IEEE International Conference on Consumer Electronics(ICCE). Piscataway, USA: IEEE, 2021: 1-3.
[31] SANYAL R, CHAKRABARTY K. Two stream deep convolutional neural network for eye state recognition and blink detection[C] //Processdings of 2019 3rd International Conference on Electronics, Materials Engineering and Nano-Technology(IEMENTech). Piscataway, USA: IEEE, 2019: 1-8.
[32] ABTAHI S, OMIDYEGANCHEH M, SHIRMOHAMMADI S, et al. YawDD: a yawning detection dataset[C] //Processdings of the 5th ACM Multimedia Systems Conference. NewYork, USA: ACM, 2014: 24-28.
[33] FEI Y, LI B, WANG H, et al. Long short-term memory network based fatigue detection with sequential mouth feature[C] //Processdings of 2020 International Symposium on Autonomous Systems(ISAS). Piscataway,USA: IEEE, 2020: 17-22.
[34] 张万枝. 机器视觉感知下的车辆主动安全技术若干问题研究[D]. 济南: 山东大学, 2015. ZHANG Wanzhi. Research on issues in vehicle active safety technology based on machine visual perception[D]. Jinan: Shandong University, 2015.
[35] YE M, ZHANG W, CAO P, et al. Driver fatigue detection based on residual channel attention network and head pose estimation[J]. Applied Sciences, 2021, 11(19): 9195-9213.
[36] CHOI I H, KIM Y G. Head pose and gaze direction tracking for detecting a drowsy driver[C] //Processdings of 2014 International Conference on Big Data and Smart Computing(BIGCOMP). Piscataway,USA: IEEE, 2014: 241-244.
[37] SRI M T, PHANINDRA P H, SAI C N, et al. Driverdrowsiness detection using eye aspect ratio(EAR), mouth aspect ratio(MAR), and driver distraction using head pose estimation[C] //Processdings of ICT Systems and Sustainability. Berlin, German: Springer, 2022: 619-627.
[38] FARHANGI F. Investigating the role of data preprocessing, hyperparameters tuning, and type of machine learning algorithm in the improvement of drowsy EEG signal modeling[J]. Intelligent Systems with Applications, 2022, 15: 200100-200108.
[39] AKINCI R, AKDOGAN E, AKTAN M E. Comparison of machine learning algorithms for recognizing drowsiness in drivers using electroencephalogram(EEG)signals[J]. International Journal of Intelligent Systems and Applications in Engineering, 2022, 10(1): 44-51.
[40] SHEN M, ZOU B, LI X, et al. Multi-source signal alignment and efficient multi-dimensional feature classification in the application of EEG-based subject-independent drowsiness detection[J]. Biomedical Signal Processing and Control, 2021, 70: 103023-103031.
[41] LEE C, AN J. LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEG[J]. Expert Systems with Applications, 2023, 213: 119032-119043.
[42] GANGADHARAN S K, VINOD A P. A nonlinear penalty driven adaptive thresholding algorithm for drowsiness detection using EEG[C] //Processdings of 2021 4th International Conference on Bio-Engineering for Smart Technologies(BioSMART). Piscataway, USA: IEEE, 2021: 1-4.
[43] AICH T K. Absent posterior alpha rhythm: an indirect indicator of seizure disorder?[J]. Indian Journal of Psychiatry, 2014, 56(1): 61-66.
[44] GROMER M, SALB D, WALZER T, et al. ECG sensor for detection of driver's drowsiness[J]. Procedia Computer Science, 2019, 159: 1938-1946.
[45] HAYAWI A A, WALEED J. Driver's drowsiness monitoring and alarming auto-system based on EOG signals[C] //Processdings of 2019 2nd International Conference on Engineering Technology and its Applications(IICETA). Piscataway, USA: IEEE, 2019: 214-218.
[46] ARTANTO D, SULISTYANTO M P, PRANOWO I D, et al. Drowsiness detection system based on eye-closure using a low-cost EMG and ESP8266[C] //Processdings of 2017 2nd International Conferences on Information Technology, Information Systems and Electrical Engineering(ICITISEE). Piscataway, USA: IEEE, 2017: 235-238.
[47] BABAEIAN M, MOZUMDAR M. Applying HRV based on line clustering method to identify driver drowsiness[C] //Processdings of 2021 IEEE 11th Annual Computing and Communication Workshop and Conference(CCWC). Piscataway, USA: IEEE, 2021: 12-21.
[48] LAWOYIN S A, FEI D Y, BAI O. A novel application of inertial measurement units(IMUs)as vehicular technologies for drowsy driving detection via steering wheel movement[J]. Open Journal of Safety Science and Technology, 2014, 4(4): 166-177.
[49] LI P, MEZIANE R, OTIS M J D, et al. Asmart safety helmet using IMU and EEG sensors for worker fatigue detection[C] //Processdings of 2014 IEEE International Symposium on Robotic and Sensors Environments(ROSE). Piscataway, USA: IEEE, 2014: 55-60.
[50] LAWOYIN S, FEI D Y, BAI O. Accelerometer-based steering-wheel movement monitoring for drowsy-driving detection[J]. Journal of Automobile Engineering, 2015, 229(2): 163-173.
[51] 毛喆. 机动车疲劳驾驶行为识别方法研究[D]. 武汉:武汉理工大学, 2011. MAO Zhe. Reaserch on identification of fatigue driving behavior[D]. Wuhan: Wuhan University of Techn-ology, 2011.
[52] ZHEN H G, DIN L, HONG Y H, et al. Driver drowsiness detection based on time series analysis of steering wheel angular velocity[C] //Processdings of 2017 9th International Conference on Measuring Technology and Mechatronics Automation(ICMTMA). Piscataway, USA: IEEE, 2017: 99-101.
[53] 屈肖蕾. 基于转向操作和车辆状态的疲劳驾驶检测方法研究[D]. 北京: 清华大学, 2013. QU Xiaolei. Detection ofdriver drowsiness based on steering operation and vehicle state[D]. Beijing: Tsinghua University, 2013.
[54] 毕雁冰. 高速汽车车道偏离预警系统可行区域感知算法研究[D]. 长春: 吉林大学, 2006. BI Yanbing. Study on algorithms of automotive highway lane departure warning system drivable area recognition[D]. Changchun: Jilin University, 2006.
[55] KATSUKI T, ZHAO K, YOSHIZUMI T. Learning to estimate driver drowsiness from car acceleration sensors using weakly labeled data[C] //Processdings of ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Piscataway, USA: IEEE, 2020: 3002-3006.
[56] 蔡素贤,杜超坎,周思毅,等. 基于车辆运行数据的疲劳驾驶状态检测[J]. 交通运输系统工程与信息, 2020,20(4): 77-82. CAI Suxian, DU Chaokan, ZHOU Siyi, et al. Fatigue driving state detection based on vehicle running data[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(4): 77-82.
[57] YNAG G, LIN Y, BHATTACHARYA P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network[J]. Information Sciences, 2010, 180(10): 1942-1954.
[58] AKIN M, KURT M B, SEZGIN N, et al. Estimating vigilance level by using EEG and EMG signals[J]. Neural Computing and Applications, 2008, 17: 227-236.
[59] BAI J, YU W, XIAO Z, et al. Two-stream spatial-temporal graph convolutional networks for driver drowsiness detection[J]. IEEE Transactions on Cybernetics, 2021, 52(12): 13821-13833.
[60] 方浩杰,董红召,林少轩,等. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报(工学版), 2023, 57(7): 1287-1296. FANG Haojie, DONG Hongzhao, LIN Shaoxuan, et al. Driver fatigue detection method based on multi-feature fusion[J]. Journal of Zhejiang University(Engineering Science), 2023, 57(7): 1287-1296.
[61] SHAH J, CHOUGULE A, CHAMOLA V, et al. Novel welch-transform based enhanced spectro-temporal analysis for cognitive microsleep detection using a single electrode EEG[J]. Neurocomputing, 2023, 549: 126387- 126399.
[62] 张弯. 基于脑电与眼电融合的便携式疲劳驾驶检测方法优化[D]. 西安: 西安理工大学, 2023. ZHANG Wan. Optimization on a portable fatigue driving fusion of EEG and detection method based on the EOG[D]. Xi'an: Xi'an University of Technology, 2023.
[63] LEE B G, CHUNG W Y. A smartphone-based driver safety monitoring system using data fusion[J]. Sensors, 2012, 12(12): 17536-17552.
[64] HA U, YOO H J. A multimodal drowsiness monitoring ear-module system with closed-loop real-time alarm[C] //Processdings of 2016 IEEE Biomedical Circuits and Systems Conference(BioCAS). Piscataway, USA: IEEE, 2016: 536-539.
[65] BOYRAZ P, ACAR M, KERR D. Multi-sensor driver drowsiness monitoring[J]. Proceedings of the Institution of Mechanical Engineers: Part D: Journal of Automobile Engineering, 2008, 222(11): 2041-2062.
[66] SUNAGAWA M, SHIKII S, NAKAI W, et al. Comprehensive drowsiness level detection model combining multimodal information[J]. IEEE Sensors Journal, 2019, 20(7): 3709-3717.
[67] JOSHI A, KYAL S, BANERJEE S, et al. In-the-wild drowsiness detection from facial expressions[C] //Processdings of 2020 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2020: 207-212.
[68] BEKHOUCHE S E, RUICHEK Y, DORNAIKA F. Driver drowsiness detection in video sequences using hybrid selection of deep features[J]. Knowledge-Based Systems, 2022, 252: 109436-109446.
[69] SUN Y, YU X. An innovative nonintrusive driver assistance system for vital signal monitoring[J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(6): 1932-1939.
[70] QIAN K, KOIKE T, NAKANURA T, et al. Learning multimodal representations for drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11539-11548.
[1] Gang HU, Lemeng WANG, Zhiyu LU, Qin WANG, Xiang XU. Importance identification method based on multi-order neighborhood hierarchical association contribution of nodes [J]. Journal of Shandong University(Engineering Science), 2024, 54(1): 1-10.
[2] Jiachun LI,Bowen LI,Jianbo CHANG. An efficient and lightweight RGB frame-level face anti-spoofing model [J]. Journal of Shandong University(Engineering Science), 2023, 53(6): 1-7.
[3] Yujiang FAN,Huanhuan HUANG,Jiaxiong DING,Kai LIAO,Binshan YU. Resilience evaluation system of the old community based on cloud model [J]. Journal of Shandong University(Engineering Science), 2023, 53(5): 1-9, 19.
[4] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[5] LIU Xing, YANG Lu, HAO Fanchang. Finger vein image retrieval based on multi-feature fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 118-126.
[6] YU Yixuan, YANG Geng, GENG Hua. Multimodal hierarchical keyframe extraction method for continuous combined motion [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 42-50.
[7] ZHANG Hao, LI Ziling, LIU Tong, ZHANG Dawei, TAO Jianhua. A technology prediction model based on fuzzy Bayesian networks with sociological factors [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 23-33.
[8] WU Yanli, LIU Shuwei, HE Dongxiao, WANG Xiaobao, JIN Di. Poisson-gamma topic model of describing multiple underlying relationships [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 51-60.
[9] YU Mingjun, DIAO Hongjun, LING Xinghong. Online multi-object tracking method based on trajectory mask [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 61-69.
[10] HUANG Huajuan, CHENG Qian, WEI Xiuxi, YU Chuchu. Adaptive crow search algorithm with Jaya algorithm and Gaussian mutation [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 11-22.
[11] LIU Fangxu, WANG Jian, WEI Benzheng. Auxiliary diagnosis algorithm for pediatric pneumonia based on multi-spatial attention [J]. Journal of Shandong University(Engineering Science), 2023, 53(2): 135-142.
[12] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
[13] Xiaobin XU,Qi WANG,Bin GAO,Zhiyu SUN,Zhongjun LIANG,Shangguang WANG. Pre-allocation of resources based on trajectory prediction in heterogeneous networks [J]. Journal of Shandong University(Engineering Science), 2022, 52(4): 12-19.
[14] Yinfeng MENG,Qingfang LI. Recognition learning based on multivariate functional principal component representation [J]. Journal of Shandong University(Engineering Science), 2022, 52(3): 1-8.
[15] Xiushan NIE,Yuling MA,Huiyan QIAO,Jie GUO,Chaoran CUI,Zhiyun YU,Xingbo LIU,Yilong YIN. Survey on student academic performance prediction from the perspective of task granularity [J]. Journal of Shandong University(Engineering Science), 2022, 52(2): 1-14.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] ZHANG Yong-hua,WANG An-ling,LIU Fu-ping . The reflected phase angle of low frequent inhomogeneous[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 22 -25 .
[3] LI Kan . Empolder and implement of the embedded weld control system[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 37 -41 .
[4] SHI Lai-shun,WAN Zhong-yi . Synthesis and performance evaluation of a novel betaine-type asphalt emulsifier[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 112 -115 .
[5] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[6] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[7] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[8] LI Liang, LUO Qiming, CHEN Enhong. Graph-based ranking model for object-level search
[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 15 -21 .
[9] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[10] WANG Bo,WANG Ning-sheng . Automatic generation and combinatory optimization of disassembly sequence for mechanical-electric assembly[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 52 -57 .