您的位置:山东大学 -> 科技期刊社 -> 《山东大学学报(工学版)》

山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (2): 1-12.doi: 10.6040/j.issn.1672-3961.0.2023.279

• 交通运输工程—智慧交通专题 •    下一篇

驾驶员疲劳驾驶检测研究综述

杨巨成1*,魏峰2,1,林亮1,贾庆祥1,刘建征1   

  1. 1.天津科技大学人工智能学院, 天津 300457;2.天津科技大学机械工程学院, 天津 300457
  • 出版日期:2024-04-20 发布日期:2024-04-17
  • 作者简介:杨巨成(1980— ),男,湖北天门人,教授,博士生导师,博士,主要研究方向为人工智能和智能驾驶. E-mail:jcyang@tust.edu.cn
  • 基金资助:
    天津市研究生科研创新资助项目(2022SKYZ370)

A research survey of driver drowsiness driving detection

YANG Jucheng1*, WEI Feng2,1, LIN Liang1, JIA Qingxiang1, LIU Jianzheng1   

  1. 1. College of Artificial Intelligence, Tianjin University of Science and Technology, Tianjin 300457, China;
    2. College of Mechanical Engineer, Tianjin University of Science and Technology, Tianjin 300457, China
  • Online:2024-04-20 Published:2024-04-17

摘要: 司机疲劳驾驶检测对于交通安全至关重要,有效的疲劳识别技术可以降低因疲劳引起的交通事故。对司机疲劳驾驶检测方法进行系统综述。介绍司机疲劳的概念及其检测的必要性,阐述疲劳驾驶行为特征并进行分类。详细总结目前广泛使用的几种疲劳驾驶公开数据集,通过归纳分析各数据集特点,对比其适用性和局限性,为后续研究提供宝贵资源。综合分析基于面部特征、生理信号特征、车辆特征以及多特征融合的司机疲劳驾驶检测方法,对比各类方法的优劣。总结司机疲劳驾驶检测领域面临的问题与挑战,对未来的发展方向进行展望。

关键词: 疲劳驾驶, 交通安全, 多特征融合, 驾驶行为, 疲劳检测

中图分类号: 

  • TP391
[1] 李都厚,刘群,袁伟,等. 疲劳驾驶与交通事故关系[J]. 交通运输工程学报, 2010, 10(2): 104-109. LI Duhou, LIU Qun, YUAN Wei, et al. Relationship betweenfatigue driving and traffic accidents[J]. Journal of Traffic and Transportation Engineering, 2010, 10(2): 104-109.
[2] National Highway Traffic Safety Adminstration. Asleep at the wheel-compendium synopsis[EB/OL].(2015-11-04)[2023-11-10]. https://www.nhtsa.gov/drowsy-driving/asleep-wheel-compendium-synopsis.
[3] KHUMPISUTH O, CHOTCHINASRI T, KOSHAKOSAI V, et al. Driver drowsiness detection using eye-closeness detection[C] //Processdings of 2016 12th International Conference on Signal-Image Technology and Internet-Based Systems(SITIS). Piscataway, USA: IEEE, 2016:661-668.
[4] BROWN I D. Driver fatigue[J]. Human Factors, 1994, 36(2): 298-314.
[5] 张旭欣,王雪松. 疲劳驾驶研究与预防最新进展[J]. 汽车与安全, 2019, 4. ZHANG Xuxin, WANG Xuesong. Latest developments in fatigue driving research and prevention[J]. Auto and Safety, 2019, 4.
[6] 孟宪超. 疲劳驾驶交通事故特点及有效预防分析[J]. 物流工程与管理, 2014, 36(8): 187-188. MENG Xianchao. Driving fatigue caused by traffic accident characteristics and effective prevention analysis[J]. Preservation of Commodities, 2014, 36(8): 187-188.
[7] CORTACERO K, FISHER T, DEMIRIS Y. RT-BENE: A dataset and baselines for real-time blink estimation in natural environments[C] //Processdings of the IEEE/CVF International Conference on Computer Vision Workshops. Piscataway, USA: IEEE, 2019: 339-357.
[8] PAN G, SUN L, WU Z, et al. Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C] //Processdings of 2007 IEEE 11th International Conference on Computer Vision. Piscataway, USA: IEEE, 2007: 1-8.
[9] SONG F, TAN X, LIU X, et al. Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients[J]. Pattern Recognition, 2014, 47(9): 2825-2838.
[10] HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild:a database for studying face recognition in unconstrained environments[C] //Processdings of Workshop on Faces in Real-Life Images: Detection, Alignment, and Recognition. California, USA: Hans Publishers, 2008: 1-15.
[11] KASSEM H A, CHOWDHURY M, ABAWAJY J, et al. Yawn based driver fatigue level prediction[J]. EPiC Series in Computing, 2020, 10(69): 372-382.
[12] WENG C H, LAI Y H, LAI S H. Driver drowsiness detection via a hierarchical temporal deep belief network[C] //Processdings of Computer Vision-ACCV 2016 Workshops. Berlin, German: Springer, 2017: 117-133.
[13] GHODDOOSIAN R, GALIB M, ATHITSOS V. A realistic dataset and baseline temporal model for early drowsiness detection[C] //Processdings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, USA: IEEE, 2019: 178-187.
[14] MASSOZ Q, LANGOHR T, FRANCOIS C, et al. The ULg multimodality drowsiness database(called DROZY)and examples of use[C] //Processdings of 2016 IEEE Winter Conference on Applications of Computer Vision(WACV). Piscataway, USA: IEEE, 2016: 1-7.
[15] ROTH M, GAVRILA D M. DD-pose-a large-scale driver head pose benchmark[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 927-934.
[16] DIAZ C K, HERN A, LOPEZ A M. A reduced feature set for driver head pose estimation[J]. Applied Soft Computing, 2016, 45: 98-107.
[17] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015:3659- 3667.
[18] BORGHI G, VENTURELLI M, VEZZANI R, et al. Poseidon: face-from-depth for driver pose estimation[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2017: 4661-4670.
[19] MARTIN M, ROITBERG A, HAURILET M, et al. Drive and act: a multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles[C] //Processdings of the IEEE/CVF International Conference on Computer Vision. Piscataway, USA: IEEE, 2019: 2801-2810.
[20] ORTEGA J D, KOSE N, CANAS P, et al. Dmd: a large-scale multi-modal driver monitoring dataset for attention and alertness analysis[C] //Processdings of Computer Vision-ECCV 2020 Workshops. Berlin, German: Springer, 2020: 387-405.
[21] YANG C, YANG Z, LI W, et al. FatigueView: amulti-camera video dataset for vision-based drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(1): 233-246.
[22] LI Y, WEI J, LIU Y, et al. Deep learning for micro-expression recognition: a survey[J]. IEEE Transactions on Affective Computing, 2022, 13(4): 2028-2046.
[23] VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C] //Processdings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2001, 1: 511-518.
[24] RAHMAN A, SIRSHAR M, KHAN A. Real time drowsiness detection using eye blink monitoring[C] //Processdings of 2015 National Software Engineering Conference(NSEC). Piscataway,USA: IEEE, 2015: 1-7.
[25] SOMMER D, GOLZ M. Evaluation of PERCLOS based current fatigue monitoring technologies[C] //Processdings of 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. Piscataway,USA: IEEE, 2010: 4456-4459.
[26] BACCOUR M H, DRIEWER F, KASNEC E, et al. Camera-based eye blink detection algorithm for assessing driver drowsiness[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 987-993.
[27] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015: 3659-3667.
[28] YAN J J, KUO H H, LIN Y F, et al. Real-time driver drowsiness detection system based on PERCLOS and grayscale image processing[C] //Processdings of 2016 International Symposium on Computer, Consumer and Control(IS3C). Piscataway,USA: IEEE, 2016: 243-246.
[29] 江跃龙,张铭智. 基于PERCLOS的列车司机疲劳检测设计与实现[J]. 计算机时代, 2023(4): 112-115. JIANG Yuelong, ZHANG Mingzhi. Design and implementation of train driver fatigue detection based on PERCLOS[J]. Computer Age, 2023(4): 112-115.
[30] KUWAHARA A, HIRAKAWA R, KAWANO H, et al. Eye fatigue prediction system using blink detection based on eye image[C] //Processdings of 2021 IEEE International Conference on Consumer Electronics(ICCE). Piscataway, USA: IEEE, 2021: 1-3.
[31] SANYAL R, CHAKRABARTY K. Two stream deep convolutional neural network for eye state recognition and blink detection[C] //Processdings of 2019 3rd International Conference on Electronics, Materials Engineering and Nano-Technology(IEMENTech). Piscataway, USA: IEEE, 2019: 1-8.
[32] ABTAHI S, OMIDYEGANCHEH M, SHIRMOHAMMADI S, et al. YawDD: a yawning detection dataset[C] //Processdings of the 5th ACM Multimedia Systems Conference. NewYork, USA: ACM, 2014: 24-28.
[33] FEI Y, LI B, WANG H, et al. Long short-term memory network based fatigue detection with sequential mouth feature[C] //Processdings of 2020 International Symposium on Autonomous Systems(ISAS). Piscataway,USA: IEEE, 2020: 17-22.
[34] 张万枝. 机器视觉感知下的车辆主动安全技术若干问题研究[D]. 济南: 山东大学, 2015. ZHANG Wanzhi. Research on issues in vehicle active safety technology based on machine visual perception[D]. Jinan: Shandong University, 2015.
[35] YE M, ZHANG W, CAO P, et al. Driver fatigue detection based on residual channel attention network and head pose estimation[J]. Applied Sciences, 2021, 11(19): 9195-9213.
[36] CHOI I H, KIM Y G. Head pose and gaze direction tracking for detecting a drowsy driver[C] //Processdings of 2014 International Conference on Big Data and Smart Computing(BIGCOMP). Piscataway,USA: IEEE, 2014: 241-244.
[37] SRI M T, PHANINDRA P H, SAI C N, et al. Driverdrowsiness detection using eye aspect ratio(EAR), mouth aspect ratio(MAR), and driver distraction using head pose estimation[C] //Processdings of ICT Systems and Sustainability. Berlin, German: Springer, 2022: 619-627.
[38] FARHANGI F. Investigating the role of data preprocessing, hyperparameters tuning, and type of machine learning algorithm in the improvement of drowsy EEG signal modeling[J]. Intelligent Systems with Applications, 2022, 15: 200100-200108.
[39] AKINCI R, AKDOGAN E, AKTAN M E. Comparison of machine learning algorithms for recognizing drowsiness in drivers using electroencephalogram(EEG)signals[J]. International Journal of Intelligent Systems and Applications in Engineering, 2022, 10(1): 44-51.
[40] SHEN M, ZOU B, LI X, et al. Multi-source signal alignment and efficient multi-dimensional feature classification in the application of EEG-based subject-independent drowsiness detection[J]. Biomedical Signal Processing and Control, 2021, 70: 103023-103031.
[41] LEE C, AN J. LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEG[J]. Expert Systems with Applications, 2023, 213: 119032-119043.
[42] GANGADHARAN S K, VINOD A P. A nonlinear penalty driven adaptive thresholding algorithm for drowsiness detection using EEG[C] //Processdings of 2021 4th International Conference on Bio-Engineering for Smart Technologies(BioSMART). Piscataway, USA: IEEE, 2021: 1-4.
[43] AICH T K. Absent posterior alpha rhythm: an indirect indicator of seizure disorder?[J]. Indian Journal of Psychiatry, 2014, 56(1): 61-66.
[44] GROMER M, SALB D, WALZER T, et al. ECG sensor for detection of driver's drowsiness[J]. Procedia Computer Science, 2019, 159: 1938-1946.
[45] HAYAWI A A, WALEED J. Driver's drowsiness monitoring and alarming auto-system based on EOG signals[C] //Processdings of 2019 2nd International Conference on Engineering Technology and its Applications(IICETA). Piscataway, USA: IEEE, 2019: 214-218.
[46] ARTANTO D, SULISTYANTO M P, PRANOWO I D, et al. Drowsiness detection system based on eye-closure using a low-cost EMG and ESP8266[C] //Processdings of 2017 2nd International Conferences on Information Technology, Information Systems and Electrical Engineering(ICITISEE). Piscataway, USA: IEEE, 2017: 235-238.
[47] BABAEIAN M, MOZUMDAR M. Applying HRV based on line clustering method to identify driver drowsiness[C] //Processdings of 2021 IEEE 11th Annual Computing and Communication Workshop and Conference(CCWC). Piscataway, USA: IEEE, 2021: 12-21.
[48] LAWOYIN S A, FEI D Y, BAI O. A novel application of inertial measurement units(IMUs)as vehicular technologies for drowsy driving detection via steering wheel movement[J]. Open Journal of Safety Science and Technology, 2014, 4(4): 166-177.
[49] LI P, MEZIANE R, OTIS M J D, et al. Asmart safety helmet using IMU and EEG sensors for worker fatigue detection[C] //Processdings of 2014 IEEE International Symposium on Robotic and Sensors Environments(ROSE). Piscataway, USA: IEEE, 2014: 55-60.
[50] LAWOYIN S, FEI D Y, BAI O. Accelerometer-based steering-wheel movement monitoring for drowsy-driving detection[J]. Journal of Automobile Engineering, 2015, 229(2): 163-173.
[51] 毛喆. 机动车疲劳驾驶行为识别方法研究[D]. 武汉:武汉理工大学, 2011. MAO Zhe. Reaserch on identification of fatigue driving behavior[D]. Wuhan: Wuhan University of Techn-ology, 2011.
[52] ZHEN H G, DIN L, HONG Y H, et al. Driver drowsiness detection based on time series analysis of steering wheel angular velocity[C] //Processdings of 2017 9th International Conference on Measuring Technology and Mechatronics Automation(ICMTMA). Piscataway, USA: IEEE, 2017: 99-101.
[53] 屈肖蕾. 基于转向操作和车辆状态的疲劳驾驶检测方法研究[D]. 北京: 清华大学, 2013. QU Xiaolei. Detection ofdriver drowsiness based on steering operation and vehicle state[D]. Beijing: Tsinghua University, 2013.
[54] 毕雁冰. 高速汽车车道偏离预警系统可行区域感知算法研究[D]. 长春: 吉林大学, 2006. BI Yanbing. Study on algorithms of automotive highway lane departure warning system drivable area recognition[D]. Changchun: Jilin University, 2006.
[55] KATSUKI T, ZHAO K, YOSHIZUMI T. Learning to estimate driver drowsiness from car acceleration sensors using weakly labeled data[C] //Processdings of ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Piscataway, USA: IEEE, 2020: 3002-3006.
[56] 蔡素贤,杜超坎,周思毅,等. 基于车辆运行数据的疲劳驾驶状态检测[J]. 交通运输系统工程与信息, 2020,20(4): 77-82. CAI Suxian, DU Chaokan, ZHOU Siyi, et al. Fatigue driving state detection based on vehicle running data[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(4): 77-82.
[57] YNAG G, LIN Y, BHATTACHARYA P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network[J]. Information Sciences, 2010, 180(10): 1942-1954.
[58] AKIN M, KURT M B, SEZGIN N, et al. Estimating vigilance level by using EEG and EMG signals[J]. Neural Computing and Applications, 2008, 17: 227-236.
[59] BAI J, YU W, XIAO Z, et al. Two-stream spatial-temporal graph convolutional networks for driver drowsiness detection[J]. IEEE Transactions on Cybernetics, 2021, 52(12): 13821-13833.
[60] 方浩杰,董红召,林少轩,等. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报(工学版), 2023, 57(7): 1287-1296. FANG Haojie, DONG Hongzhao, LIN Shaoxuan, et al. Driver fatigue detection method based on multi-feature fusion[J]. Journal of Zhejiang University(Engineering Science), 2023, 57(7): 1287-1296.
[61] SHAH J, CHOUGULE A, CHAMOLA V, et al. Novel welch-transform based enhanced spectro-temporal analysis for cognitive microsleep detection using a single electrode EEG[J]. Neurocomputing, 2023, 549: 126387- 126399.
[62] 张弯. 基于脑电与眼电融合的便携式疲劳驾驶检测方法优化[D]. 西安: 西安理工大学, 2023. ZHANG Wan. Optimization on a portable fatigue driving fusion of EEG and detection method based on the EOG[D]. Xi'an: Xi'an University of Technology, 2023.
[63] LEE B G, CHUNG W Y. A smartphone-based driver safety monitoring system using data fusion[J]. Sensors, 2012, 12(12): 17536-17552.
[64] HA U, YOO H J. A multimodal drowsiness monitoring ear-module system with closed-loop real-time alarm[C] //Processdings of 2016 IEEE Biomedical Circuits and Systems Conference(BioCAS). Piscataway, USA: IEEE, 2016: 536-539.
[65] BOYRAZ P, ACAR M, KERR D. Multi-sensor driver drowsiness monitoring[J]. Proceedings of the Institution of Mechanical Engineers: Part D: Journal of Automobile Engineering, 2008, 222(11): 2041-2062.
[66] SUNAGAWA M, SHIKII S, NAKAI W, et al. Comprehensive drowsiness level detection model combining multimodal information[J]. IEEE Sensors Journal, 2019, 20(7): 3709-3717.
[67] JOSHI A, KYAL S, BANERJEE S, et al. In-the-wild drowsiness detection from facial expressions[C] //Processdings of 2020 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2020: 207-212.
[68] BEKHOUCHE S E, RUICHEK Y, DORNAIKA F. Driver drowsiness detection in video sequences using hybrid selection of deep features[J]. Knowledge-Based Systems, 2022, 252: 109436-109446.
[69] SUN Y, YU X. An innovative nonintrusive driver assistance system for vital signal monitoring[J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(6): 1932-1939.
[70] QIAN K, KOIKE T, NAKANURA T, et al. Learning multimodal representations for drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11539-11548.
[1] 赵涛,张宁,王小超,马川义,田源,张圣涛,杨梓梁. 基于图神经网络轨迹预测的合流区交通冲突预测方法[J]. 山东大学学报 (工学版), 2024, 54(2): 36-46.
[2] 宋修广,赵涛,毕研美,张紫豪,杜聪,田源,孔晓光. 光纤传感技术在道路交通中的应用[J]. 山东大学学报 (工学版), 2024, 54(2): 13-26.
[3] 刘行,杨璐,郝凡昌. 基于多特征融合的手指静脉图像检索方法[J]. 山东大学学报 (工学版), 2023, 53(2): 118-126.
[4] 蔡念, 张国宏, 楼朋旭, 戴青云. 基于形状和纹理的外观设计专利图像检索方法[J]. 山东大学学报(工学版), 2011, 41(2): 1-4.
[5] 吕国仁,闫书明,白书锋,贾 宁,马 亮 . 高速公路新型波形梁护栏端头实车碰撞性能研究[J]. 山东大学学报(工学版), 2008, 38(4): 47-52 .
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 王素玉,艾兴,赵军,李作丽,刘增文 . 高速立铣3Cr2Mo模具钢切削力建模及预测[J]. 山东大学学报(工学版), 2006, 36(1): 1 -5 .
[2] 张永花,王安玲,刘福平 . 低频非均匀电磁波在导电界面的反射相角[J]. 山东大学学报(工学版), 2006, 36(2): 22 -25 .
[3] 李 侃 . 嵌入式相贯线焊接控制系统开发与实现[J]. 山东大学学报(工学版), 2008, 38(4): 37 -41 .
[4] 施来顺,万忠义 . 新型甜菜碱型沥青乳化剂的合成与性能测试[J]. 山东大学学报(工学版), 2008, 38(4): 112 -115 .
[5] 孔祥臻,刘延俊,王勇,赵秀华 . 气动比例阀的死区补偿与仿真[J]. 山东大学学报(工学版), 2006, 36(1): 99 -102 .
[6] 来翔 . 用胞映射方法讨论一类MKdV方程[J]. 山东大学学报(工学版), 2006, 36(1): 87 -92 .
[7] 余嘉元1 , 田金亭1 , 朱强忠2 . 计算智能在心理学中的应用[J]. 山东大学学报(工学版), 2009, 39(1): 1 -5 .
[8] 李梁,罗奇鸣,陈恩红. 对象级搜索中基于图的对象排序模型(英文)[J]. 山东大学学报(工学版), 2009, 39(1): 15 -21 .
[9] 陈瑞,李红伟,田靖. 磁极数对径向磁轴承承载力的影响[J]. 山东大学学报(工学版), 2018, 48(2): 81 -85 .
[10] 王波,王宁生 . 机电装配体拆卸序列的自动生成及组合优化[J]. 山东大学学报(工学版), 2006, 36(2): 52 -57 .