Journal of Shandong University(Engineering Science) ›› 2021, Vol. 51 ›› Issue (2): 19-25.doi: 10.6040/j.issn.1672-3961.0.2020.225

• Machine Learning & Data Mining • Previous Articles     Next Articles

Real-time semantic segmentation of high-resolution remote sensing image based on multi-level feature cascade

Chunhong CAO(),Hongxuan DUAN*(),Ling CAO,Lele ZHANG,Kai HU,Fen XIAO   

  1. The MOE Key Laboratory of Intelligent Computing & Information Processing (Xiangtan University), Xiangtan 411100, Hunan, China
  • Received:2020-06-17 Online:2021-04-20 Published:2021-04-16
  • Contact: Hongxuan DUAN E-mail:caoch@xtu.edu.cn;201821562036@smail.xtu.edu.cn

RichHTML

43

PDF (PC)

696

Abstract:

Aiming at the problems of long segmentation time and inaccurate segmentation of small targets in remote sensing image semantic segmentation, a fast semantic segmentation model of high-resolution remote sensing image based on multi-level feature cascade network (MFCNet) was proposed. The model was mainly composed of feature encoding, feature fusion and target refinement. Feature encoding extracted the input images feature of different resolutions and used different backbone networks. Due to the lower resolution of low-resolution images, heavy-weight backbone networks were used to obtain rich semantic information with fewer parameters. For medium and high-resolution images, lightweight backbone network was used to reduce the amount of parameters and obtain global information. While medium and low-resolution encoding used the way of weights and calculation sharing to further reduce model parameters and computational complexity. The feature fusion section fused features from different branches to obtain information at different scales. The target refinement used residual to correction the fused features and the features of the coded part to restore the spatial detail information of the image, making the segmentation more accurate. And the entire model worked efficiently in an end-to-end manner. The experimental verified the validity of the model in semantic segmentation of remote sensing images, and achieved a good balance between model complexity and accuracy.

Key words: remote sensing image, real-time semantic segmentation, multi-level feature fusion, feature extraction, end-to-end

CLC Number: 

  • TP18

Fig.1

Network model based on multi-level feature cascade"

Fig.2

Module of cascade feature fusion"

Fig.3

Scheme of residual correction"

Fig.4

Comparison results of different segmentation methods under the Vaihingen dataset"

Fig.5

Comparison results of different segmentation methods under the Potsdam dataset"

Table 1

Comparison results of MFCNet and other methods under Vaihingen dataset  %"

方法 F1 OA mIOU
不透水的表面 建筑物 低植被 树木 汽车 平均
UNet 85.14 89.24 73.21 82.12 71.84 80.31 83.82 67.88
ICNet 85.84 90.36 72.71 81.99 36.86 73.55 83.68 61.69
PSPNet 85.68 90.13 73.46 83.02 65.40 79.53 84.52 67.20
MFCNet 86.90 91.02 75.26 83.54 74.15 82.18 85.53 70.53

Table 2

Comparison results of MFCNet and other methods under Potsdam dataset  %"

方法 F1 OA mIOU
不透水的表面 建筑物 低植被 树木 汽车 平均
UNet 88.16 89.74 80.68 80.36 88.90 85.57 86.04 75.12
ICNet 88.49 90.10 78.22 77.73 79.75 82.86 85.19 71.20
PSPNet 90.04 93.59 82.08 80.89 88.00 86.92 87.89 77.25
MFCNet 89.84 92.98 81.37 79.76 90.82 86.91 87.47 77.39

Table 3

Comparison of complexity under Vaihingen dataset"

方法 分割图像的平均时间/s 参数量
UNet 26.38 7846822
ICNet 64.85 6743442
PSPNet 31.28 46692053
MFCNet 34.17 7795320

Table 4

Comparison of complexity under Potsdam dataset"

方法 分割图像的平均时间/s 参数量
UNet 194.55 7846822
ICNet 908.18 6743442
PSPNet 220.13 46692053
MFCNet 210.40 7795320

Table 5

Influence of different branch connection positions on experimental results  %"

分支 OA F1 mIOU
分支1 81.70 77.02 63.72
分支2 85.53 82.18 70.53
1 刘航, 汪西莉. 基于注意力机制的遥感图像分割模型[J]. 激光与光电子学进展, 2020, 57 (4): 170- 180.
LIU Hang , WANG Xili . Remote sensing image segmentation model based on attention mechanism[J]. Laser & Optoelectronics Progress, 2020, 57 (4): 170- 180.
2 LIU C , YUEN J , TORRALBA A . Nonparametric scene parsing via label transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (12): 2368- 2382.
doi: 10.1109/TPAMI.2011.131
3 LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39 (4): 640- 651.
4 BADRINARAYANAN V , KENDALL A , CIPOLLA R . Segnet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39 (12): 2481- 2495.
5 ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017: 2881-2890.
6 CHEN L, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]// Proceedings of the International Conference on Learning Representations (ICLR). San Diego, USA: IEEE, 2015: 1-14.
7 CHEN L , PAPANDREOU G , KOKKINOS I , et al. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848.
doi: 10.1109/TPAMI.2017.2699184
8 CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 801-818.
9 NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 1520-1528.
10 MEHTA S, RASTEGARI M, SHAPIRO L, et al. Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). California, USA: IEEE, 2019: 9190-9200.
11 WANG Y, ZHOU Q, LIU J, et al. Lednet: a lightweight encoder-decoder network for real-time semantic segmentation[C]//Proceedings of the IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019: 1860-1864.
12 YU C, WANG J, PENG C, et al. Bisenet: bilateral segmentation network for real-time semantic segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 334-349.
13 LI H, XIONG P, FAN H, et al. Dfanet: deep feature aggregation for real-time semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). California, USA: IEEE, 2019: 9522-9531.
14 HE T, SHEN C, TIAN Z, et al. Knowledge adaptation for efficient semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). California, USA: IEEE, 2019: 578-587.
15 ZHAO H, QI X, SHEN X, et al. Icnet for real-time semantic segmentation on high-resolution images[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 405-420.
16 ZHUANG J, YANG J, GU L, et al. Shelfnet for fast semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW). Seoul, Korea: IEEE, 2019: 847-856.
17 MARIUS C, MOHAMED O, SEBASTIAN R, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). Las Vegas, USA: IEEE, 2016: 3213-3223.
18 LIN T, MICHAEL M, SERGE B, James Hays, et al. Microsoft coco: common objects in context[C]// Proceedings of the European Conference on Computer Vision (ECCV). Zurich, Switzerland: Springer, 2014: 740-755.
19 OLAF R, PHILIPP F, THOMAS B. U-net: convolutional networks for biomedical image segmentation[C]// Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MICCA). Munich, Germany: Springer, 2015: 234-241.
20 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 770-778.
[1] Ying LI,Jiankun WANG. The classification of mild cognitive impairment based on supervised graph regularization and information fusion [J]. Journal of Shandong University(Engineering Science), 2023, 53(4): 65-73.
[2] Donglan LIU,Xin LIU,Jianfei CHEN,Wenting WANG,Hao ZHANG,Lei MA,Dong LI. End-to-end security encryption scheme of NB-IoT for smart grid based on physical unclonable function [J]. Journal of Shandong University(Engineering Science), 2020, 50(1): 63-71.
[3] Jun FAN,Qiaolin YE,Ning YE. Face recognition based on improved prameter-free supervised localitypreserving projections [J]. Journal of Shandong University(Engineering Science), 2019, 49(1): 10-16.
[4] Guoxin WANG,Fengdong CHEN,Guodong LIU. Feature extraction method of color pseudo-random coded structured light [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 55-60.
[5] YE Ziyun, YANG Jinfeng. A finger-vein recognition method based on weighted graph model [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 103-109.
[6] ZHANG Zhenyue, LI Fei, JIANG Mingyan. Unsupervised face image feature extraction based on low-rank representation projection [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(1): 15-20.
[7] YU Qingmin, LI Xiaolei, ZHAI Yong. Feature extraction method of rolling bearing inner ring in wind turbine based on improved EMD and feature box [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(3): 89-95.
[8] GUO Chao, YANG Yan, JIANG Yongquan, SONG Yi. Condition recognition of high-speed train based on multi-view classification ensemble [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 7-14.
[9] ZHANG Yuling, YIN Chuanhuan. Android malware detection based on SVM [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2017, 47(1): 42-47.
[10] ZHONG Zhiyan, WEN Zhiqiang, ZHANG Xiaoyun, YE Degang. Neighborhood similarity descriptor used in halftone image [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2016, 46(3): 58-64.
[11] XIAO Qiao, PEI Jihong, WANG Lixia, GONG Zhicheng. Ship detection in remote sensing image based on the fuzzy fusion of multi-channel Gabor filtering [J]. Journal of Shandong University(Engineering Science), 2015, 45(5): 29-35.
[12] LU Dan, ZHOU Yiqi. Vibration analysis of excavator seat based on EEMD and CWT [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(3): 58-64.
[13] ZHOU Zhe, SHANG Lin. A sentiment analysis method based on dynamic lexicon and three-way decision [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2015, 45(1): 19-23.
[14] YU Hai-jing1,2, LI Gui-ju1*. Color smoke image recognition based on differential box-counting fractal dimension algorithm [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(1): 35-40.
[15] LI Hui1,2, HU Yun1,3, LI Cun-hua1. The technique of gas disaster information feature extraction based on rough set theory [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2012, 42(5): 91-95.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] ZHANG Yong-hua,WANG An-ling,LIU Fu-ping . The reflected phase angle of low frequent inhomogeneous[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 22 -25 .
[2] KONG Xiang-zhen,LIU Yan-jun,WANG Yong,ZHAO Xiu-hua . Compensation and simulation for the deadband of the pneumatic proportional valve[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 99 -102 .
[3] LAI Xiang . The global domain of attraction for a kind of MKdV equations[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(1): 87 -92 .
[4] YU Jia yuan1, TIAN Jin ting1, ZHU Qiang zhong2. Computational intelligence and its application in psychology[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 1 -5 .
[5] JI Tao,GAO Xu/sup>,SUN Tong-jing,XUE Yong-duan/sup>,XU Bing-yin/sup> . Characteristic analysis of fault generated traveling waves in 10 Kv automatic blocking and continuous power transmission lines[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(2): 111 -116 .
[6] QIN Tong, SUN Fengrong*, WANG Limei, WANG Qinghao, LI Xincai. 3D surface reconstruction using the shape based interpolation guided by maximal discs[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 1 -5 .
[7] Yue Khing Toh1， XIAO Wendong2， XIE Lihua1. Wireless sensor network for distributed target tracking: practices via real test bed development[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 50 -56 .
[8] SUN Dianzhu, ZHU Changzhi, LI Yanrui. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 84 -86 .
[9] HU Tian-liang,LI Peng,ZHANG Cheng-rui,ZUO Yi . Design of a QEP decode counter based on VHDL[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(3): 10 -13 .
[10] BO De-Yun, ZHANG Dao-Jiang. Adaptive spectral clustering algorithm[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 22 -26 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd