Journal of Shandong University(Engineering Science) ›› 2020, Vol. 50 ›› Issue (2): 100-107.doi: 10.6040/j.issn.1672-3961.0.2019.424

• Machine Learning & Data Mining • Previous Articles     Next Articles

Fire detection based on lightweight convolutional neural network

Yunyang YAN1,2,3(),Chenxi DU1,2,Yian LIU2,Shangbing GAO1   

  1. 1. Faculty of Computer & Software Engineering, Huaiyin Institute of Technology, Huaian 223003, Jiangsu, China
    2. School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, Jiangsu, China
    3. School of Computer Engineering, Jiangsu Ocean University, Lianyungang 222005, Jiangsu, China
  • Received:2019-07-25 Online:2020-04-20 Published:2020-04-16
  • Supported by:
    国家自然科学基金资助项目(61402192);江苏省“六大人才高峰”项目(2013DZXX-023);江苏省“青蓝工程”;淮安市“533英才工程”

RichHTML

11

PDF (PC)

491

Abstract:

A novel lightweight flame detection method was proposed based on MobileNet. The video flame detection rate was promoted by the feature receptive field of DCB(dilated convolution block)module expand based on depthwise separable convolution and dilated convolution to strengthen the feature semantic information. The SSD(single shot multibox detector) detection framework was also optimized. The lightweight detection model DMSSD(Dilated MobileNet-SSD) was provided. Experiments showed that the mean average precision was increased by 1.7% and 3.8% respectively on the PASCAL VOC dataset and the VisiFire dataset of Bilkent University. Furthermore, the detection speed was up to 80 frames per second. The robustness and real-time performance of DMSSD were strong.

Key words: fire detection, MobileNet, dilated convolution, channel shuffle, DCB

CLC Number: 

  • TP391

Fig.1

Structure of MobileNet-SSD model"

Fig.2

Comparison of standard convolution anddilated convolution"

Table 1

Receptive field of convolution layers"

层级 输入尺寸 卷积核尺寸 步长 输出尺寸 感受野
Conv5 38×38 1 1 38×38 43
Conv11 19×19 1 1 19×19 219
Conv13 10×10 1 1 10×10 315
Conv14-2 10×10 3 2 5×5 379
Conv15-2 5×5 3 2 3×3 507
Conv16-2 3×3 3 2 2×2 763
Conv17-2 2×2 3 2 1×1 1 275

Fig.3

Illustration of the gridding problem"

Fig.4

Channel shuffle"

Fig.5

Skip connections"

Fig.6

Dilated convolution block"

Fig.7

Structure of DMSSD model"

Table 2

Effectiveness of various designs onthe VOC 2007 test"

方法 移除Conv16 复合膨胀卷积 通道重排 残差连接 mAP/%
MobileNet 72.7
对照组1 √ 73.0
对照组2 √ √ 74.0
对照组3 √ √ √ 74.2
DMSSD √ √ √ √ 74.4

Table 3

Results on PASCAL VOC 2007"

方法 MACs/
109		 Parameters/
106		 GPU Inference/
ms		 mAP/
%
Tiny-YOLO[16] 3.49 15.86 6.85 57.1
DSOD_smallest[17] 5.29 5.90 18.74 73.6
Pelee[18] 1.21 5.43 14.17 76.4
MobileNet-SSD[1] 1.16 5.77 5.92 72.7
DMSSD 1.25 5.76 6.50 74.4

Table 4

Single class test results on PASCAL VOC 2007"

方法 飞机 自行车 鸟 船 瓶子 大巴 轿车 猫 椅子 奶牛 桌子 狗 马 摩托 人 植物 绵羊 沙发 火车 电视
MobileNet-SSD[1] 73.9 82.4 71.1 61.2 39.1 82.6 80.2 88.2 53.8 67.8 78.4 80.8 87.9 85.6 76.5 43.4 65.0 79.4 86.7 69.6
DMSSD 74.7 83.8 71.3 59.7 40.8 84.6 81.0 88.6 56.8 72.6 77.9 83.2 89.2 86.5 77.3 45.5 71.8 79.7 87.8 75.6

Table 5

Number of samples in the VisiFire dataset"

类别 训练集 测试集 合计
图片 5 224 2 582 7 806
目标 10 920 4 130 15 050

Table 6

Detection results on VisiFire"

模型 MACs/
109		 Parameters/
106		 FPS/
(帧·s-1)		 mAP/
%
MobileNet-SSD[1] 1.13 5.52 84 74.3
DMSSD 1.22 5.49 80 78.1

Fig.8

Comparison of model performance"

Fig.9

Video examples and detection results"

Table 7

Detection results on flame videos"

视频 总帧数 MobileNet-SSD[1] Tiny-YOLO[16] Pelee[18] Tiny-DSOD[19] DMSSD
TP/% FP/% TP/% FP/% TP/% FP/% TP/% FP/% TP/% FP/%
Video1 200 96.0 4.0 94.5 5.5 98.5 1.5 98.0 2.0 97.5 2.5
Video2 216 95.8 4.2 96.8 3.2 98.1 1.9 97.7 2.3 97.2 2.8
Video3 439 90.7 9.3 84.1 15.9 93.2 6.8 91.8 8.2 92.5 7.5
Video4 170 94.7 5.3 97.1 2.9 97.6 2.4 95.3 4.7 96.5 3.5
Video5 595 92.9 7.1 84.5 15.5 96.1 3.9 92.4 7.6 95.0 5.0
Video6 470 92.1 7.9 72.3 27.7 94.7 5.3 91.5 8.8 91.9 8.1
平均 348.3 93.7 6.3 88.2 11.8 96.4 3.6 94.5 5.5 95.1 4.9

Table 8

Detection results on non-flame videos"

视频 总帧数 MobileNet-SSD[1] Tiny-YOLO[16] Pelee[18] Tiny-DSOD[19] DMSSD
TP/% FP/% TP/% FP/% TP/% FP/% TP/% FP/% TP/% FP/%
Video7 144 93.1 6.9 85.4 14.6 95.1 4.9 92.4 7.6 93.8 6.2
Video8 120 93.3 6.7 91.6 8.4 95.0 5.0 94.2 5.8 95.0 5.0
Video9 228 96.9 3.1 94.7 5.3 97.8 2.2 96.1 3.9 97.4 2.6
平均 164 94.4 5.6 90.6 9.4 96.0 4.0 94.2 5.8 95.4 4.6
1 HOWARD A G, ZHU Menglong, CHEN Bo, et al. Mobilenets: efficient convolutional neural networks for mobile vision applications[J]. arXiv preprint arXiv: 1704.04861, 2017. https://arxiv.org/abs/1704.04861.
2 CHENG Yu, WANG Duo, ZHOU Pan, et al. A survey of model compression and acceleration for deep neural networks[J]. arXiv preprint arXiv: 1710.09282, 2017. https://arxiv.org/abs/1710.09282.
3 HAN Song, MAO Huizi, DALLY W J. Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding[J]. arXiv preprint arXiv: 1510.00149, 2015. https://arxiv.org/abs/1510.00149.
4 LUO Ping, ZHU Zhenyao, LIU Ziwei, et al. Face model compression by distilling knowledge from neurons[C]//Proceedings of 30th AAAI Conference on Artificial Intelligence. Menlo Park, USA: Association for the Advancement of Artificial Intelligence, 2016: 3560-3566.
5 JADERBERG M, VEDALDI A, ZISSERMAN A. Speeding up convolutional neural networks with low rank expansions[J]. arXiv preprint arXiv: 1405.3866, 2014. https://arxiv.org/abs/1405.3866.
6 SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Computer Society, 2016: 2818-2826.
7 CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE Computer Society, 2017: 1800-1807.
8 ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. Shufflenet: an extremely efficient convolutional neural network for mobile devices[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society, 2018: 6848-6856.
9 CHEN Liangchieh, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J]. arXiv preprint arXiv: 1706.05587, 2017. https://arxiv.org/abs/1706.05587.
10 DUMOULIN V, VISIN F. A guide to convolution arithmetic for deep learning[J]. arXiv preprint arXiv: 1603.07285, 2016. https://arxiv.org/abs/1603.07285.
11 WEI S E, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE Computer Society, 2016: 4724-4732.
12 CAO Xudong. A practical theory for designing very deep convolutional neural network[R/OL].[2018-08-01]. https://www.kaggle.com/c/datasciencebowl/discussion/13166.
13 YU Fisher, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. arXiv preprint arXiv: 1511.07122, 2015. https://arxiv.org/abs/1511.07122.
14 WANG Panqu, CHEN Pengfei, YUAN Ye, et al. Understanding convolution for semantic segmentation[C]//2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Nevada, USA: IEEE Computer Society, 2018: 1451-1460.
15 CHEN Liangchieh, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[J]. arXiv preprint arXiv: 1802.02611, 2018. https://arxiv.org/abs/1802.02611
16 REDMON J, FARHADI A. Yolo9000: better, faster, stronger[C]//Computer Vision and Pattern Recognition. Washington, USA: IEEE Computer Society, 2017: 6517-6525.
17 SHEN Zhiqiang, LIU Zhuang, LI Jianguo, et al. Dsod: learning deeply supervised object detectors from scratch[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice, Italia: IEEE Computer Society, 2017: 1919-1927.
18 WANG R J, LI Xiang, LING C X. Pelee: a real-time object detection system on mobile devices[C]//Advances in Neural Information Processing Systems. Montreal, Canada: Neural Information Processing Sys-tems Foundation, Inc., 2018: 1963-1972.
19 LI Yuxi, LI Jiuwei, LIN Weiyao, et al. Tiny-dsod: lightweight object detection for resource-restricted usages[J]. arXiv preprint arXiv: 1807.11013, 2018. https://arxiv.org/abs/1807.11013.
20 ZOPH B, VASUDEVAN V, SHLENS J, et al. Learning transferable architectures for scalable image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE Computer Society, 2018: 8697-8710.
[1] Shengnan ZHANG,Lei WANG,Chunhong CHANG,Benli HAO. Image denoising based on 3D shearlet transform and BM4D [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 83-90.
[2] Longmao HU,Xuegang HU. Identification of the same product feature based on multi-dimension similarity and sentiment word expansion [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 50-59.
[3] Yanping CHEN,Li FENG,Yongbin QIN,Ruizhang HUANG. A syntactic element recognition method based on deep neural network [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 44-49.
[4] Wei YAN,Damin ZHANG,Huijuan ZHANG,Ziyun XI,Zhongyun CHEN. Improved bird swarm algorithms based on mixed decision making [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 34-43.
[5] Shiqi SONG,Yan PIAO,Zexin JIANG. Vehicle classification and tracking for complex scenes based on improved YOLOv3 [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 27-33.
[6] Ningning CHEN,Jianwei ZHAO,Zhenghua ZHOU. Visual tracking algorithm based on verifying networks [J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 17-26.
[7] Yuenan ZHAO,Guiyou CHEN,Chen SUN,Ning LU,Liwei LIAO. Risk assessment method based on spatial hidden danger distribution and motion intention analysis [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 28-34.
[8] Jialin SU,Yuanzhuo WANG,Xiaolong JIN,Xueqi CHENG. Entity alignment method based on adaptive attribute selection [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 14-20.
[9] Guoyong CAI,Qiang LIN,Kaiqi REN. Cross-domain text sentiment classification based on domain-adversarialnetwork and BERT [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 1-7,20.
[10] Yuanxi YAO. Analysis of wind power convergence trend quantitation based on sub-scene reconstruction [J]. Journal of Shandong University(Engineering Science), 2019, 49(6): 86-92.
[11] Ji ZHANG,Cui JIN,Hongyuan WANG,Shoubing CHEN. Pedestrian recognition based on singular value decomposition pedestrian alignment network [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 91-97.
[12] Junmei YUE,Dongmei ZHANG. Lightweight self-adaptive CSI-based positioning algorithm in underground mine [J]. Journal of Shandong University(Engineering Science), 2019, 49(5): 112-118.
[13] Zongtang ZHANG,Sen WANG,Shilin SUN. An ensemble learning algorithm for unbalanced data classification [J]. Journal of Shandong University(Engineering Science), 2019, 49(4): 8-13.
[14] Xindi CHEN,Tianrui LI,Huanhuan YANG. Visualization of interactive ThemeRiver based on time-series data [J]. Journal of Shandong University(Engineering Science), 2019, 49(4): 29-35, 43.
[15] Jinchao HUANG. Object tracking algorithm based on deep residual features and entropy energy optimization [J]. Journal of Shandong University(Engineering Science), 2019, 49(4): 14-23.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] WANG Su-yu,<\sup>,AI Xing<\sup>,ZHAO Jun<\sup>,LI Zuo-li<\sup>,LIU Zeng-wen<\sup> . Milling force prediction model for highspeed end milling 3Cr2Mo steel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 1 -5 .
[2] YU Hai-bo,LI Yu,YU Tian,LEI Hong . Influence of the dimensions of W-band folded waveguide slow-wave system on its cold characteristics[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 90 -94 .
[3] LONG Zhi-Jian, ZHANG Chang-Qiao. Synthesis and properties of associating DRA by binary copolymerization based on lauryl methacrylate[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 128 -132 .
[4] LI Xin-Ping, DAI Yi-Fei, HU Jing. Fluid-solid coupling analysis of surrounding rock mass stability and water inflow forecast of a tunnel in a karst zone[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 1 -6 .
[5] GAO Yang, ZHANG Qing-Song, YUAN Xiao-Shuai, XU Zhen-Hao, LIU Bin. Application of geological radar to geological forecast in karst tunnel[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 82 -86 .
[6] ZHAI Xin-Xian, CHEN Dong-Hai, GUO Nian-Bo, GOU Pan-Feng. The laws of strata behavior for the gob-side tailgate in the  Jining No.3 Coal Mine[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 92 -96 .
[7] QIAO Xiaoyan. Automatic recognition method of microscopic image of harmful algae[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 1 -6 .
[8]

L〖AKU¨] Guo-ren,YAN Shu-ming,BAI Shu-feng,JIA Ning,MA Liang

. Impact property research on a new type of guardrail terminal for freeway[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(4): 47 -52 .
[9] LI Jie ，LIU Hong. A method of fractal artistic pattern generation based on a genetic algorithm[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(6): 33 -36 .
[10] LI Chun-xiao， YUE Qin-yan， LU Lei， GAO Bao-yu， YANG Zhong-lian， SI Xiao-hui， NI Shou-qing， WANG Yuan-fang. Synthesis and application of  hydrophobically  associating  cationic  polyacrylamide    [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(6): 99 -104 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd