山东大学学报 (工学版) ›› 2024, Vol. 54 ›› Issue (2): 1-12.doi: 10.6040/j.issn.1672-3961.0.2023.279
• 交通运输工程—智慧交通专题 • 下一篇
杨巨成1*,魏峰2,1,林亮1,贾庆祥1,刘建征1
YANG Jucheng1*, WEI Feng2,1, LIN Liang1, JIA Qingxiang1, LIU Jianzheng1
摘要： 司机疲劳驾驶检测对于交通安全至关重要,有效的疲劳识别技术可以降低因疲劳引起的交通事故。对司机疲劳驾驶检测方法进行系统综述。介绍司机疲劳的概念及其检测的必要性,阐述疲劳驾驶行为特征并进行分类。详细总结目前广泛使用的几种疲劳驾驶公开数据集,通过归纳分析各数据集特点,对比其适用性和局限性,为后续研究提供宝贵资源。综合分析基于面部特征、生理信号特征、车辆特征以及多特征融合的司机疲劳驾驶检测方法,对比各类方法的优劣。总结司机疲劳驾驶检测领域面临的问题与挑战,对未来的发展方向进行展望。
中图分类号:
| [1] 李都厚,刘群,袁伟,等. 疲劳驾驶与交通事故关系[J]. 交通运输工程学报, 2010, 10(2): 104-109. LI Duhou, LIU Qun, YUAN Wei, et al. Relationship betweenfatigue driving and traffic accidents[J]. Journal of Traffic and Transportation Engineering, 2010, 10(2): 104-109. [2] National Highway Traffic Safety Adminstration. Asleep at the wheel-compendium synopsis[EB/OL].(2015-11-04)[2023-11-10]. https://www.nhtsa.gov/drowsy-driving/asleep-wheel-compendium-synopsis. [3] KHUMPISUTH O, CHOTCHINASRI T, KOSHAKOSAI V, et al. Driver drowsiness detection using eye-closeness detection[C] //Processdings of 2016 12th International Conference on Signal-Image Technology and Internet-Based Systems(SITIS). Piscataway, USA: IEEE, 2016:661-668. [4] BROWN I D. Driver fatigue[J]. Human Factors, 1994, 36(2): 298-314. [5] 张旭欣,王雪松. 疲劳驾驶研究与预防最新进展[J]. 汽车与安全, 2019, 4. ZHANG Xuxin, WANG Xuesong. Latest developments in fatigue driving research and prevention[J]. Auto and Safety, 2019, 4. [6] 孟宪超. 疲劳驾驶交通事故特点及有效预防分析[J]. 物流工程与管理, 2014, 36(8): 187-188. MENG Xianchao. Driving fatigue caused by traffic accident characteristics and effective prevention analysis[J]. Preservation of Commodities, 2014, 36(8): 187-188. [7] CORTACERO K, FISHER T, DEMIRIS Y. RT-BENE: A dataset and baselines for real-time blink estimation in natural environments[C] //Processdings of the IEEE/CVF International Conference on Computer Vision Workshops. Piscataway, USA: IEEE, 2019: 339-357. [8] PAN G, SUN L, WU Z, et al. Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C] //Processdings of 2007 IEEE 11th International Conference on Computer Vision. Piscataway, USA: IEEE, 2007: 1-8. [9] SONG F, TAN X, LIU X, et al. Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients[J]. Pattern Recognition, 2014, 47(9): 2825-2838. [10] HUANG G B, MATTAR M, BERG T, et al. Labeled faces in the wild:a database for studying face recognition in unconstrained environments[C] //Processdings of Workshop on Faces in Real-Life Images: Detection, Alignment, and Recognition. California, USA: Hans Publishers, 2008: 1-15. [11] KASSEM H A, CHOWDHURY M, ABAWAJY J, et al. Yawn based driver fatigue level prediction[J]. EPiC Series in Computing, 2020, 10(69): 372-382. [12] WENG C H, LAI Y H, LAI S H. Driver drowsiness detection via a hierarchical temporal deep belief network[C] //Processdings of Computer Vision-ACCV 2016 Workshops. Berlin, German: Springer, 2017: 117-133. [13] GHODDOOSIAN R, GALIB M, ATHITSOS V. A realistic dataset and baseline temporal model for early drowsiness detection[C] //Processdings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, USA: IEEE, 2019: 178-187. [14] MASSOZ Q, LANGOHR T, FRANCOIS C, et al. The ULg multimodality drowsiness database(called DROZY)and examples of use[C] //Processdings of 2016 IEEE Winter Conference on Applications of Computer Vision(WACV). Piscataway, USA: IEEE, 2016: 1-7. [15] ROTH M, GAVRILA D M. DD-pose-a large-scale driver head pose benchmark[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 927-934. [16] DIAZ C K, HERN A, LOPEZ A M. A reduced feature set for driver head pose estimation[J]. Applied Soft Computing, 2016, 45: 98-107. [17] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015:3659- 3667. [18] BORGHI G, VENTURELLI M, VEZZANI R, et al. Poseidon: face-from-depth for driver pose estimation[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2017: 4661-4670. [19] MARTIN M, ROITBERG A, HAURILET M, et al. Drive and act: a multi-modal dataset for fine-grained driver behavior recognition in autonomous vehicles[C] //Processdings of the IEEE/CVF International Conference on Computer Vision. Piscataway, USA: IEEE, 2019: 2801-2810. [20] ORTEGA J D, KOSE N, CANAS P, et al. Dmd: a large-scale multi-modal driver monitoring dataset for attention and alertness analysis[C] //Processdings of Computer Vision-ECCV 2020 Workshops. Berlin, German: Springer, 2020: 387-405. [21] YANG C, YANG Z, LI W, et al. FatigueView: amulti-camera video dataset for vision-based drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(1): 233-246. [22] LI Y, WEI J, LIU Y, et al. Deep learning for micro-expression recognition: a survey[J]. IEEE Transactions on Affective Computing, 2022, 13(4): 2028-2046. [23] VIOLA P, JONES M. Rapid object detection using a boosted cascade of simple features[C] //Processdings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway,USA: IEEE, 2001, 1: 511-518. [24] RAHMAN A, SIRSHAR M, KHAN A. Real time drowsiness detection using eye blink monitoring[C] //Processdings of 2015 National Software Engineering Conference(NSEC). Piscataway,USA: IEEE, 2015: 1-7. [25] SOMMER D, GOLZ M. Evaluation of PERCLOS based current fatigue monitoring technologies[C] //Processdings of 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. Piscataway,USA: IEEE, 2010: 4456-4459. [26] BACCOUR M H, DRIEWER F, KASNEC E, et al. Camera-based eye blink detection algorithm for assessing driver drowsiness[C] //Processdings of 2019 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2019: 987-993. [27] TZIMIROPOULOS G. Project-out cascaded regression with an application to face alignment[C] //Processdings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, USA: IEEE, 2015: 3659-3667. [28] YAN J J, KUO H H, LIN Y F, et al. Real-time driver drowsiness detection system based on PERCLOS and grayscale image processing[C] //Processdings of 2016 International Symposium on Computer, Consumer and Control(IS3C). Piscataway,USA: IEEE, 2016: 243-246. [29] 江跃龙,张铭智. 基于PERCLOS的列车司机疲劳检测设计与实现[J]. 计算机时代, 2023(4): 112-115. JIANG Yuelong, ZHANG Mingzhi. Design and implementation of train driver fatigue detection based on PERCLOS[J]. Computer Age, 2023(4): 112-115. [30] KUWAHARA A, HIRAKAWA R, KAWANO H, et al. Eye fatigue prediction system using blink detection based on eye image[C] //Processdings of 2021 IEEE International Conference on Consumer Electronics(ICCE). Piscataway, USA: IEEE, 2021: 1-3. [31] SANYAL R, CHAKRABARTY K. Two stream deep convolutional neural network for eye state recognition and blink detection[C] //Processdings of 2019 3rd International Conference on Electronics, Materials Engineering and Nano-Technology(IEMENTech). Piscataway, USA: IEEE, 2019: 1-8. [32] ABTAHI S, OMIDYEGANCHEH M, SHIRMOHAMMADI S, et al. YawDD: a yawning detection dataset[C] //Processdings of the 5th ACM Multimedia Systems Conference. NewYork, USA: ACM, 2014: 24-28. [33] FEI Y, LI B, WANG H, et al. Long short-term memory network based fatigue detection with sequential mouth feature[C] //Processdings of 2020 International Symposium on Autonomous Systems(ISAS). Piscataway,USA: IEEE, 2020: 17-22. [34] 张万枝. 机器视觉感知下的车辆主动安全技术若干问题研究[D]. 济南: 山东大学, 2015. ZHANG Wanzhi. Research on issues in vehicle active safety technology based on machine visual perception[D]. Jinan: Shandong University, 2015. [35] YE M, ZHANG W, CAO P, et al. Driver fatigue detection based on residual channel attention network and head pose estimation[J]. Applied Sciences, 2021, 11(19): 9195-9213. [36] CHOI I H, KIM Y G. Head pose and gaze direction tracking for detecting a drowsy driver[C] //Processdings of 2014 International Conference on Big Data and Smart Computing(BIGCOMP). Piscataway,USA: IEEE, 2014: 241-244. [37] SRI M T, PHANINDRA P H, SAI C N, et al. Driverdrowsiness detection using eye aspect ratio(EAR), mouth aspect ratio(MAR), and driver distraction using head pose estimation[C] //Processdings of ICT Systems and Sustainability. Berlin, German: Springer, 2022: 619-627. [38] FARHANGI F. Investigating the role of data preprocessing, hyperparameters tuning, and type of machine learning algorithm in the improvement of drowsy EEG signal modeling[J]. Intelligent Systems with Applications, 2022, 15: 200100-200108. [39] AKINCI R, AKDOGAN E, AKTAN M E. Comparison of machine learning algorithms for recognizing drowsiness in drivers using electroencephalogram(EEG)signals[J]. International Journal of Intelligent Systems and Applications in Engineering, 2022, 10(1): 44-51. [40] SHEN M, ZOU B, LI X, et al. Multi-source signal alignment and efficient multi-dimensional feature classification in the application of EEG-based subject-independent drowsiness detection[J]. Biomedical Signal Processing and Control, 2021, 70: 103023-103031. [41] LEE C, AN J. LSTM-CNN model of drowsiness detection from multiple consciousness states acquired by EEG[J]. Expert Systems with Applications, 2023, 213: 119032-119043. [42] GANGADHARAN S K, VINOD A P. A nonlinear penalty driven adaptive thresholding algorithm for drowsiness detection using EEG[C] //Processdings of 2021 4th International Conference on Bio-Engineering for Smart Technologies(BioSMART). Piscataway, USA: IEEE, 2021: 1-4. [43] AICH T K. Absent posterior alpha rhythm: an indirect indicator of seizure disorder?[J]. Indian Journal of Psychiatry, 2014, 56(1): 61-66. [44] GROMER M, SALB D, WALZER T, et al. ECG sensor for detection of driver's drowsiness[J]. Procedia Computer Science, 2019, 159: 1938-1946. [45] HAYAWI A A, WALEED J. Driver's drowsiness monitoring and alarming auto-system based on EOG signals[C] //Processdings of 2019 2nd International Conference on Engineering Technology and its Applications(IICETA). Piscataway, USA: IEEE, 2019: 214-218. [46] ARTANTO D, SULISTYANTO M P, PRANOWO I D, et al. Drowsiness detection system based on eye-closure using a low-cost EMG and ESP8266[C] //Processdings of 2017 2nd International Conferences on Information Technology, Information Systems and Electrical Engineering(ICITISEE). Piscataway, USA: IEEE, 2017: 235-238. [47] BABAEIAN M, MOZUMDAR M. Applying HRV based on line clustering method to identify driver drowsiness[C] //Processdings of 2021 IEEE 11th Annual Computing and Communication Workshop and Conference(CCWC). Piscataway, USA: IEEE, 2021: 12-21. [48] LAWOYIN S A, FEI D Y, BAI O. A novel application of inertial measurement units(IMUs)as vehicular technologies for drowsy driving detection via steering wheel movement[J]. Open Journal of Safety Science and Technology, 2014, 4(4): 166-177. [49] LI P, MEZIANE R, OTIS M J D, et al. Asmart safety helmet using IMU and EEG sensors for worker fatigue detection[C] //Processdings of 2014 IEEE International Symposium on Robotic and Sensors Environments(ROSE). Piscataway, USA: IEEE, 2014: 55-60. [50] LAWOYIN S, FEI D Y, BAI O. Accelerometer-based steering-wheel movement monitoring for drowsy-driving detection[J]. Journal of Automobile Engineering, 2015, 229(2): 163-173. [51] 毛喆. 机动车疲劳驾驶行为识别方法研究[D]. 武汉:武汉理工大学, 2011. MAO Zhe. Reaserch on identification of fatigue driving behavior[D]. Wuhan: Wuhan University of Techn-ology, 2011. [52] ZHEN H G, DIN L, HONG Y H, et al. Driver drowsiness detection based on time series analysis of steering wheel angular velocity[C] //Processdings of 2017 9th International Conference on Measuring Technology and Mechatronics Automation(ICMTMA). Piscataway, USA: IEEE, 2017: 99-101. [53] 屈肖蕾. 基于转向操作和车辆状态的疲劳驾驶检测方法研究[D]. 北京: 清华大学, 2013. QU Xiaolei. Detection ofdriver drowsiness based on steering operation and vehicle state[D]. Beijing: Tsinghua University, 2013. [54] 毕雁冰. 高速汽车车道偏离预警系统可行区域感知算法研究[D]. 长春: 吉林大学, 2006. BI Yanbing. Study on algorithms of automotive highway lane departure warning system drivable area recognition[D]. Changchun: Jilin University, 2006. [55] KATSUKI T, ZHAO K, YOSHIZUMI T. Learning to estimate driver drowsiness from car acceleration sensors using weakly labeled data[C] //Processdings of ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). Piscataway, USA: IEEE, 2020: 3002-3006. [56] 蔡素贤,杜超坎,周思毅,等. 基于车辆运行数据的疲劳驾驶状态检测[J]. 交通运输系统工程与信息, 2020,20(4): 77-82. CAI Suxian, DU Chaokan, ZHOU Siyi, et al. Fatigue driving state detection based on vehicle running data[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(4): 77-82. [57] YNAG G, LIN Y, BHATTACHARYA P. A driver fatigue recognition model based on information fusion and dynamic Bayesian network[J]. Information Sciences, 2010, 180(10): 1942-1954. [58] AKIN M, KURT M B, SEZGIN N, et al. Estimating vigilance level by using EEG and EMG signals[J]. Neural Computing and Applications, 2008, 17: 227-236. [59] BAI J, YU W, XIAO Z, et al. Two-stream spatial-temporal graph convolutional networks for driver drowsiness detection[J]. IEEE Transactions on Cybernetics, 2021, 52(12): 13821-13833. [60] 方浩杰,董红召,林少轩,等. 多特征融合的驾驶员疲劳状态检测方法[J]. 浙江大学学报(工学版), 2023, 57(7): 1287-1296. FANG Haojie, DONG Hongzhao, LIN Shaoxuan, et al. Driver fatigue detection method based on multi-feature fusion[J]. Journal of Zhejiang University(Engineering Science), 2023, 57(7): 1287-1296. [61] SHAH J, CHOUGULE A, CHAMOLA V, et al. Novel welch-transform based enhanced spectro-temporal analysis for cognitive microsleep detection using a single electrode EEG[J]. Neurocomputing, 2023, 549: 126387- 126399. [62] 张弯. 基于脑电与眼电融合的便携式疲劳驾驶检测方法优化[D]. 西安: 西安理工大学, 2023. ZHANG Wan. Optimization on a portable fatigue driving fusion of EEG and detection method based on the EOG[D]. Xi'an: Xi'an University of Technology, 2023. [63] LEE B G, CHUNG W Y. A smartphone-based driver safety monitoring system using data fusion[J]. Sensors, 2012, 12(12): 17536-17552. [64] HA U, YOO H J. A multimodal drowsiness monitoring ear-module system with closed-loop real-time alarm[C] //Processdings of 2016 IEEE Biomedical Circuits and Systems Conference(BioCAS). Piscataway, USA: IEEE, 2016: 536-539. [65] BOYRAZ P, ACAR M, KERR D. Multi-sensor driver drowsiness monitoring[J]. Proceedings of the Institution of Mechanical Engineers: Part D: Journal of Automobile Engineering, 2008, 222(11): 2041-2062. [66] SUNAGAWA M, SHIKII S, NAKAI W, et al. Comprehensive drowsiness level detection model combining multimodal information[J]. IEEE Sensors Journal, 2019, 20(7): 3709-3717. [67] JOSHI A, KYAL S, BANERJEE S, et al. In-the-wild drowsiness detection from facial expressions[C] //Processdings of 2020 IEEE Intelligent Vehicles Symposium(IV). Piscataway, USA: IEEE, 2020: 207-212. [68] BEKHOUCHE S E, RUICHEK Y, DORNAIKA F. Driver drowsiness detection in video sequences using hybrid selection of deep features[J]. Knowledge-Based Systems, 2022, 252: 109436-109446. [69] SUN Y, YU X. An innovative nonintrusive driver assistance system for vital signal monitoring[J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(6): 1932-1939. [70] QIAN K, KOIKE T, NAKANURA T, et al. Learning multimodal representations for drowsiness detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11539-11548. |
| [1] | 赵涛,张宁,王小超,马川义,田源,张圣涛,杨梓梁. 基于图神经网络轨迹预测的合流区交通冲突预测方法[J]. 山东大学学报 (工学版), 2024, 54(2): 36-46. |
| [2] | 宋修广,赵涛,毕研美,张紫豪,杜聪,田源,孔晓光. 光纤传感技术在道路交通中的应用[J]. 山东大学学报 (工学版), 2024, 54(2): 13-26. |
| [3] | 刘行,杨璐,郝凡昌. 基于多特征融合的手指静脉图像检索方法[J]. 山东大学学报 (工学版), 2023, 53(2): 118-126. |
| [4] | 蔡念, 张国宏, 楼朋旭, 戴青云. 基于形状和纹理的外观设计专利图像检索方法[J]. 山东大学学报(工学版), 2011, 41(2): 1-4. |
| [5] | 吕国仁,闫书明,白书锋,贾 宁,马 亮 . 高速公路新型波形梁护栏端头实车碰撞性能研究[J]. 山东大学学报(工学版), 2008, 38(4): 47-52 . |
|
