Journal of Shandong University(Engineering Science) ›› 2023, Vol. 53 ›› Issue (6): 1-7.doi: 10.6040/j.issn.1672-3961.0.2023.132

• Machine Learning & Data Mining •     Next Articles

An efficient and lightweight RGB frame-level face anti-spoofing model

Jiachun LI(),Bowen LI,Jianbo CHANG   

  1. School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, Guangdong, China
  • Received:2023-06-21 Online:2023-12-20 Published:2023-12-19

RichHTML

7

PDF (PC)

203

Abstract:

Based on the challenge when deploying a deep learning-based face anti-spoofing (FAS) algorithm on general-purpose consumer devices with only RGB camera, an efficient and lightweight RGB frame-level FAS model (EL-FAS) was proposed. To improve the generalization ability of the model and achieve high performance under constrained conditions, a novel global spatial self-attention mechanism was explored to capture global feature dependencies, and an equal-channel pixel-wise binary supervision method was designed to force our model to learn shared features from different pixels. The Bottlenecks residual block was used to establish the backbone network to reduce parameters. Analysis and the experimental results showed that EL-FAS model achieved state-of-the-art performance in the OULU-NPU dataset, obtained competitive performance in the SiW dataset and cross-dataset tests. The model was lightweight, with only 1.34×106 parameters.

Key words: deep learning, face anti-spoofing, self-attention mechanism, pixel-wise supervision, lightweight model

CLC Number: 

  • TP391

Fig.1

Network architecture of EL-FAS"

Table 1

Network architecture details of EL-FAS"

解编码器 模块 输入 操作 t c s
编码器 E1 3×2242 Conv3×3 32 2
32×1122 Bottleneck 1 16 1
E2 16×562 Bottleneck 6 24 2
24×562 Bottleneck 6 24 1
24×562 Bottleneck 6 32 2
E3 32×282 Bottleneck 6 32 1
32×282 Bottleneck 6 32 1
32×282 Bottleneck 6 64 2
64×142 Bottleneck 6 64 1
64×142 Bottleneck 6 64 1
E4 64×142 Bottleneck 6 64 1
64×142 Bottleneck 6 96 1
96×142 Bottleneck 6 96 1
96×142 Bottleneck 6 96 1
解码器 96×142 ConvTrans 32 2
D1 64×282 Bottleneck 6 32 1
32×282 Bottleneck 6 32 1
32×282 ConvTrans 24 2
D2 48×562 Bottleneck 6 24 1
24×562 Bottleneck 6 24 1
24×562 ConvTrans 16 2
D3 32×1122 Bottleneck 6 16 1
16×1122 Bottleneck 6 16 1
D4 16×1122 ConvTrans 16 2
16×2242 Conv3×3 3 1

Fig.2

Framework of global spatial self-attention module"

Table 2

The summary of the datasets used"

数据集 用户数 视频数 攻击设备 测试协议数
SiW 165 4 478 iPad, iPhone7, GalaxyS8, Asus MB168B(RGB) 3
OULU-NPU 55 4 950 Dell 1905FP, MacBook(RGB) 4
Replay-Attack 50 1 200 iPad, iPhone 3GS(RGB) 1
CASIA-FASD 50 600 A4 Paper, iPad(RGB) 1

Table 3

The ablation study results for intra-dataset test  单位: %"

模型 RAPCE RBPCE RACE
Single 4.2±9.3 4.6±6.8 4.4±4.9
Equal 3.3±4.7 5.4±6.8 4.4±3.2
Single+Attention 3.4±4.5 4.4±6.8 3.9±3.5
Equal+Attention 1.6±3.7 5.8±8.3 3.7±4.0

Table 4

The ablation study results for inter-dataset test  单位: %"

模型 RAPCE RBPCE RACE
Single 3.3±3.7 2.5±2.5 2.9±2.7
Equal 3.0±4.0 2.5±2.2 2.8±2.0
Single+Attention 3.0±4.0 2.0±1.9 2.5±1.8
Equal+Attention 0.8±1.8 3.7±4.3 2.3±2.4

Table 5

The intra-dataset test results on OULU-NPU  单位: %"

协议 方法 RAPCE RBPCE RACE
1 GRADIANT[18] 1.3 12.5 6.9
Auxiliary[17] 1.6 1.6 1.6
FaceDs[19] 1.2 1.7 1.5
STDN[20] 0.8 1.3 1.1
DSGT[5] 2.0 0 1.0
DeepPix[10] 0.8 0 0.4
EL-FAS 0 1.6 0.8
2 DeepPix[10] 11.4 0.6 6.0
FaceDs[19] 4.2 4.4 4.3
Auxiliary[17] 2.7 2.7 2.7
GRADIANT[18] 3.1 1.9 2.5
DSGT[5] 2.5 1.3 1.9
STDN[20] 2.3 1.6 1.9
EL-FAS 0.3 0.5 0.4
3 DeepPix[10] 11.7±19.6 10.6±14.1 11.1±9.4
GRADIANT[18] 2.6±3.9 5.0±5.3 3.8±2.4
Auxiliary[17] 2.6±3.9 5.0±5.3 3.8±2.4
FaceDs[19] 4.0±1.8 3.8±1.2 3.6±1.6
STDN[20] 1.6±1.6 4.0±5.4 2.8±3.3
DSGT[5] 3.2±2.0 2.2±1.4 2.7±0.6
EL-FAS 1.9±2.4 3.5±2.4 2.7±1.2
4 DeepPix[10] 36.7±29.7 13.3±16.8 25.0±12.7
GRADIANT[18] 5.0±4.5 15.0±7.1 10.0±5.0
Auxiliary[17] 9.3±5.6 10.4±6.0 9.5±6.0
FaceDs[19] 5.1±6.3 6.1±5.1 5.6±5.7
DSGT[5] 6.7±7.5 3.3±4.1 5.0±2.2
STDN[20] 2.3±3.6 5.2±5.4 3.8±4.2
EL-FAS 1.6±3.7 5.8±8.3 3.7±4.0

Table 6

The intra-dataset test results on SiW  单位: %"

协议 方法 RAPCE RBPCE RACE
1 Auxiliary[17] 3.58 3.58 3.58
DSGT[5] 0.64 0.17 0.40
STDN[20] 0 0 0
EL-FAS 0 0 0
2 Auxiliary[17] 0.57±0.69 0.57±0.69 0.57±0.69
DSGT[5] 0 0.04±0.08 0.02±0.04
STDN[20] 0 0 0
EL-FAS 0.08±0.14 0 0.04±0.07
3 Auxiliary[17] 8.31±3.81 8.31±3.80 8.31±3.81
STDN[20] 8.33±3.33 7.50±3.33 7.90±3.30
DSGT[5] 2.63±3.72 2.92±3.42 2.78±3.57
EL-FAS 25.80±23.50 0 12.90±11.70

Table 7

The inter-dataset test results between CASIA-FASD and Replay-Attack  单位: % "

方法 RHTE
CR RC
CNN[3] 48.5 45.5
FaceDs[19] 28.5 41.1
Auxiliary[16] 27.6 28.4
EL-FAS 31.2 31.1

Table 8

The inter-dataset test results from SiW to OULU-NPU dataset  单位: %"

协议 方法 RAPCE RBPCE RACE
1 Auxiliary[17] 10.0
DSGT[5] 1.7 13.3 7.5
EL-FAS 3.3 4.6 3.9
2 Auxiliary[17] 14.1
DSGT[5] 9.7 14.2 11.9
EL-FAS 3.1 20.5 11.8
3 Auxiliary[17] 13.8±5.7
DSGT[5] 17.5±4.6 11.7±12.0 14.6±4.8
EL-FAS 3.9±2.8 10.8±3.8 7.4±2.5
4 Auxiliary[17] 10.0±8.8
DSGT[5] 0.8±1.9 10.0±11.6 5.4±5.7
EL-FAS 0.8±1.8 3.7±4.3 2.3±2.4

Table 9

The lightweight indicator comparison  单位: 个"

方法 参数量 浮点运算量
LSTM-CNN[4] 2.07×106 4.77×109
DSGT[5] 5.53×106 773.40×109
Depth[8] 9.25×106 20.57×109
BASN[9] 80.40×106 274.13×109
DeepPix[10] 3.20×106 4.62×109
EL-FAS 1.34×106 0.79×109
1 PATEL K , HAN H , JAIN A K . Secure face unlock: spoof detection on smartphones[J]. IEEE Transactions on Information Forensics and Security, 2016, 11 (10): 2268- 2283.
doi: 10.1109/TIFS.2016.2578288
2 KOMULAINEN J, HADID A, PIETIKÄINEN M. Context based face anti-spoofing[C]//Proceedings of the 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS). Arlington, USA: IEEE, 2013: 6712690.
3 YANG J, LEI Z, LI S Z. Learn convolutional neural network for face anti-spoofing[EB/OL]. (2014-08-26)[2023-02-21]. https://arxiv.org/abs/1408.5601.
4 XU Z, LI S, DENG W. Learning temporal features using LSTM-CNN architecture for face anti-spoofing[C]//Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition(ACPR). Kuala Lumpur, Malaysia: IEEE, 2015: 141-145.
5 WANG Z, YU Z, ZHAO C, et al. Deep spatial gradient and temporal depth learning for face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 5042-5051.
6 ZHANG S, WANG X, LIU A, et al. A dataset and benchmark for large-scale multi-modal face anti-spoofing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 919-928.
7 WANG Z , WANG Q , DENG W , et al. Learning multi-granularity temporal characteristics for face anti-spoofing[J]. IEEE Transactions on Information Forensics and Security, 2022, 17, 1254- 1269.
doi: 10.1109/TIFS.2022.3158062
8 ATOUM Y, LIU Y, JOURABLOO A, et al. Face anti-spoofing using patch and depth-based CNNs[C]//Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB). Denver, USA: IEEE, 2017: 319-328.
9 KIM T, KIM Y H, KIM I, et al. BASN: enriching feature representation using bipartite auxiliary supervisions for face anti-spoofing[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Seoul, Korea: IEEE, 2019: 494-503.
10 GEORGE A, MARCEL S. Deep pixel-wise binary supervision for face presentation attack detection[C]// Proceedings of the 2019 International Conference on Biometrics (ICB). Crete, Greece: IEEE, 2019: 19352833.
11 RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer, 2015: 234-241.
12 SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 4510-4520.
13 VISIN F, KASTNER K, CHO K, et al. ReNet: a recurrent neural network based alternative to convolutional networks[EB/OL]. (2015-07-23)[2023-02-21]. https://arxiv.org/abs/1505.00393.
14 BOULKENAFET Z, KOMULAINEN J, LI L, et al. OULU-NPU: a mobile face presentation attack database with real-world variations[C]//Proceedings of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). Washington, USA: IEEE, 2017: 612-618.
15 CHINGOVSKA I, ANJOS A, MARCEL S. On the effectiveness of local binary patterns in face anti-spoofing[C]//2012 BIOSIG-Proceedings of the International Conference of Biometrics Special Interest Group (BIOSIG). Darmstadt, Germany: IEEE, 2012: 13029854.
16 ZHANG Z, YAN J, LIU S, et al. A face anti-spoofing database with diverse attacks[C]//Proceedings of the 2012 5th IAPR International Conference on Biometrics (ICB). New Delhi, India: IEEE, 2012: 26-31.
17 LIU Y, JOURABLOO A, LIU X. Learning deep models for face anti-spoofing: binary or auxiliary supervision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 389-398.
18 BOULKENAFET Z, KOMULAINEN J, AKHTAR Z, et al. A competition on generalized software-based face presentation attack detection in mobile scenarios[C]//Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB). Denver, USA: IEEE, 2017: 688-696.
19 JOURABLOO A, LIU Y, LIU X. Face de-spoofing: anti-spoofing via noise modeling[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: IEEE, 2018: 290-306.
20 LIU Y, STEHOUWER J, LIU X. On disentangling spoof trace for generic face anti-spoofing[C]//Proceedings of the European Conference on Computer Vision. Glasgow, England: Springer, 2020: 406-422.
[1] Yue YUAN,Yanli WANG,Kan LIU. Named entity recognition model based on dilated convolutional block architecture [J]. Journal of Shandong University(Engineering Science), 2022, 52(6): 105-114.
[2] Tongyu JIANG, Fan CHEN, Hongjie HE. Lightweight face super-resolution network based on asymmetric U-pyramid reconstruction [J]. Journal of Shandong University(Engineering Science), 2022, 52(1): 1-8.
[3] Jianqing WU,Xiuguang SONG. Review on development of simultaneous localization and mapping technology [J]. Journal of Shandong University(Engineering Science), 2021, 51(5): 16-31.
[4] Qingfa CHAI,Shoujing SUN,Jifu QIU,Ming CHEN,Zhen WEI,Wei CONG. Prediction method of power grid emergency supplies under meteorological disasters [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 76-83.
[5] YANG Xiuyuan, PENG Tao, YANG Liang, LIN Hongfei. Adaptive multi-domain sentiment analysis based on knowledge distillation [J]. Journal of Shandong University(Engineering Science), 2021, 51(3): 15-21.
[6] LIAO Jinping, MO Yuchang, YAN Ke. Model and application of short-term electricity consumption forecast based on C-LSTM [J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 90-97.
[7] LIU Shuai, WANG Lei, DING Xutao. Emotional EEG recognition based on Bi-LSTM [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 35-39.
[8] Guoyong CAI,Xinhao HE,Yangyang CHU. Visual sentiment analysis based on spatial attention mechanism and convolutional neural network [J]. Journal of Shandong University(Engineering Science), 2020, 50(4): 8-13.
[9] Chunyang LI,Nan LI,Tao FENG,Zhuhe WANG,Jingkai MA. Abnormal sound detection of washing machines based on deep learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 108-117.
[10] Delei CHEN, Cheng WANG, Jianwei CHEN, Yiyin WU. GRU-based collaborative filtering recommendation algorithm with active learning [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 21-27.
[11] Ji ZHANG,Cui JIN,Hongyuan WANG,Shoubing CHEN. Pedestrian recognition based on singular value decomposition pedestrian alignment network [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 91-97.
[12] Peng WAN. Object detection of 3D point clouds based on F-PointNet [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 98-104.
[13] Zhixiang LIANG,Xiaoming LIU,Ying MU,Yutian LIU. Prediction method of wind power and PV ramp event based on deep learning [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 24-28.
[14] Yutian LIU, Runjia SUN, Hongtao WANG, Xueping GU. Review on application of artificial intelligence in power system restoration [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 1-8.
[15] Lizhao LI, Guoyong CAI, Jiao PAN. A microblog rumor events detection method based on C-GRU [J]. Journal of Shandong University(Engineering Science), 2019, 49(2): 102-106.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[2] CHOU Wu-Sheng, WANG Shuo. Study on the adaptive algorithm of the force reflection robotic master under large stiffness of the environment[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(1): 1 -5 .
[3] LI Guo-zheng1, SHI Miao-jing1, LI Fu-feng2, WANG Yi-qin2. An empirical study on tongue image detection[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 87 -95 .
[4] DENG Bin,WANG Jiang . Estimating parameters of a neuron model based on chaos synchronization and adaptive control[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 19 -23 .
[5] GAO Hou-Lei, TIAN Jia, DU Jiang, WU Zhi-Gang, LIU Chu-Min. Distributed generation—new technology in energy development[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 106 -110 .
[6] ZHANG Jianming, LIU Quansheng, TANG Zhicheng, ZHAN Ting, JIANG Yalong. New peak shear strength criterion with inclusion of shear action history[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 0, (): 77 -81 .
[7] JIA Chao,ZHAO Jian-yu,XU Bang-shu,YUE Chang-cheng,LI Shu-chen . Research on rock soil liquefaction of the Qingshui railway tunnel under dynamic vibration load[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 83 -87 .
[8] JI Hui, SHANG Qiang-Sen, SHU Hai-Bei, CUI Xin-Zhuang, LIU Chao. Analysis of the key influencing factors of structure layer stress of asphalt pavement in goaf[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(6): 121 -124 .
[9] YUN Ru-an1,2, DONG Zeng-chuan1, WANG Hao-fang2. Multiobjective optimization of a reservoir based on NSGA2[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(6): 124 -128 .
[10] LEI Xiao-feng1, ZHUANG Wei1, CHENG Yu1, DING Shi-fei1, XIE Kun-qing2. OPHCLUS:An order-preserving based hierarchical clustering algorithm[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 48 -55 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd