基于多级特征级联的遥感图像实时语义分割

doi:10.6040/j.issn.1672-3961.0.2020.225

摘要/Abstract

摘要：

针对遥感图像语义分割中存在的分割耗时长、分割小目标不准确的问题，提出基于多级特征级联的高分辨率遥感图像快速语义分割模型(multi-level feature cascade network, MFCNet)。该模型主要由特征编码、特征融合以及目标细化3部分组成。特征编码对输入的不同分辨率图像用不同量级主干网络进行特征提取，由于低分辨率图像分辨率较低，使用重量级的主干网络在增加较少参数的情况下可以获取丰富的语义信息，而中、高分辨图像分辨率较大，使用轻量级主干网络既减少参数量又可获取全局信息。中等和低分辨率的编码使用权重和计算共享的方式，进一步减少模型参数，降低计算复杂性。特征融合对来自不同分支的特征进行融合，以获取不同尺度的信息。目标细化采用残差校正对融合后的特征和编码部分的特征进行融合校正，以恢复图像的空间细节信息，使分割更加准确。该模型可以端到端的方式有效地工作，试验验证所提模型在遥感图像语义分割中的有效性，在模型复杂性和精度上取得较好的平衡。

关键词: 遥感图像, 实时语义分割, 多级特征融合, 特征提取, 端到端

Abstract:

Aiming at the problems of long segmentation time and inaccurate segmentation of small targets in remote sensing image semantic segmentation, a fast semantic segmentation model of high-resolution remote sensing image based on multi-level feature cascade network (MFCNet) was proposed. The model was mainly composed of feature encoding, feature fusion and target refinement. Feature encoding extracted the input images feature of different resolutions and used different backbone networks. Due to the lower resolution of low-resolution images, heavy-weight backbone networks were used to obtain rich semantic information with fewer parameters. For medium and high-resolution images, lightweight backbone network was used to reduce the amount of parameters and obtain global information. While medium and low-resolution encoding used the way of weights and calculation sharing to further reduce model parameters and computational complexity. The feature fusion section fused features from different branches to obtain information at different scales. The target refinement used residual to correction the fused features and the features of the coded part to restore the spatial detail information of the image, making the segmentation more accurate. And the entire model worked efficiently in an end-to-end manner. The experimental verified the validity of the model in semantic segmentation of remote sensing images, and achieved a good balance between model complexity and accuracy.

Key words: remote sensing image, real-time semantic segmentation, multi-level feature fusion, feature extraction, end-to-end

中图分类号:

TP18

曹春红,段鸿轩,曹玲,张乐乐,胡凯,肖芬. 基于多级特征级联的遥感图像实时语义分割[J]. 山东大学学报 (工学版), 2021, 51(2): 19-25.

Chunhong CAO,Hongxuan DUAN,Ling CAO,Lele ZHANG,Kai HU,Fen XIAO. Real-time semantic segmentation of high-resolution remote sensing image based on multi-level feature cascade[J]. Journal of Shandong University(Engineering Science), 2021, 51(2): 19-25.

图/表 10

图1

图2

图3

图4

图5

表1

表2

表3

表4

表5

参考文献 20

1	刘航, 汪西莉. 基于注意力机制的遥感图像分割模型[J]. 激光与光电子学进展, 2020, 57 (4): 170- 180.
	LIU Hang , WANG Xili . Remote sensing image segmentation model based on attention mechanism[J]. Laser & Optoelectronics Progress, 2020, 57 (4): 170- 180.
2	LIU C , YUEN J , TORRALBA A . Nonparametric scene parsing via label transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (12): 2368- 2382. doi: 10.1109/TPAMI.2011.131
3	LONG J , SHELHAMER E , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014, 39 (4): 640- 651.
4	BADRINARAYANAN V , KENDALL A , CIPOLLA R . Segnet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39 (12): 2481- 2495.
5	ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017: 2881-2890.
6	CHEN L, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]// Proceedings of the International Conference on Learning Representations (ICLR). San Diego, USA: IEEE, 2015: 1-14.
7	CHEN L , PAPANDREOU G , KOKKINOS I , et al. Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40 (4): 834- 848. doi: 10.1109/TPAMI.2017.2699184
8	CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 801-818.
9	NOH H, HONG S, HAN B. Learning deconvolution network for semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015: 1520-1528.
10	MEHTA S, RASTEGARI M, SHAPIRO L, et al. Espnetv2: a light-weight, power efficient, and general purpose convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). California, USA: IEEE, 2019: 9190-9200.
11	WANG Y, ZHOU Q, LIU J, et al. Lednet: a lightweight encoder-decoder network for real-time semantic segmentation[C]//Proceedings of the IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019: 1860-1864.
12	YU C, WANG J, PENG C, et al. Bisenet: bilateral segmentation network for real-time semantic segmentation[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 334-349.
13	LI H, XIONG P, FAN H, et al. Dfanet: deep feature aggregation for real-time semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). California, USA: IEEE, 2019: 9522-9531.
14	HE T, SHEN C, TIAN Z, et al. Knowledge adaptation for efficient semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). California, USA: IEEE, 2019: 578-587.
15	ZHAO H, QI X, SHEN X, et al. Icnet for real-time semantic segmentation on high-resolution images[C]// Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018: 405-420.
16	ZHUANG J, YANG J, GU L, et al. Shelfnet for fast semantic segmentation[C]//Proceedings of the IEEE International Conference on Computer Vision Workshop (ICCVW). Seoul, Korea: IEEE, 2019: 847-856.
17	MARIUS C, MOHAMED O, SEBASTIAN R, et al. The cityscapes dataset for semantic urban scene understanding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recogintion (CVPR). Las Vegas, USA: IEEE, 2016: 3213-3223.
18	LIN T, MICHAEL M, SERGE B, James Hays, et al. Microsoft coco: common objects in context[C]// Proceedings of the European Conference on Computer Vision (ECCV). Zurich, Switzerland: Springer, 2014: 740-755.
19	OLAF R, PHILIPP F, THOMAS B. U-net: convolutional networks for biomedical image segmentation[C]// Proceedings of the Medical Image Computing and Computer-Assisted Intervention (MICCA). Munich, Germany: Springer, 2015: 234-241.
20	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 770-778.

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

方法	F1						OA	mIOU
方法	不透水的表面	建筑物	低植被	树木	汽车	平均	OA	mIOU
UNet	85.14	89.24	73.21	82.12	71.84	80.31	83.82	67.88
ICNet	85.84	90.36	72.71	81.99	36.86	73.55	83.68	61.69
PSPNet	85.68	90.13	73.46	83.02	65.40	79.53	84.52	67.20
MFCNet	86.90	91.02	75.26	83.54	74.15	82.18	85.53	70.53

方法	F1						OA	mIOU
方法	不透水的表面	建筑物	低植被	树木	汽车	平均	OA	mIOU
UNet	88.16	89.74	80.68	80.36	88.90	85.57	86.04	75.12
ICNet	88.49	90.10	78.22	77.73	79.75	82.86	85.19	71.20
PSPNet	90.04	93.59	82.08	80.89	88.00	86.92	87.89	77.25
MFCNet	89.84	92.98	81.37	79.76	90.82	86.91	87.47	77.39

方法	分割图像的平均时间/s	参数量
UNet	26.38	7846822
ICNet	64.85	6743442
PSPNet	31.28	46692053
MFCNet	34.17	7795320

方法	分割图像的平均时间/s	参数量
UNet	194.55	7846822
ICNet	908.18	6743442
PSPNet	220.13	46692053
MFCNet	210.40	7795320

分支	OA	F1	mIOU
分支1	81.70	77.02	63.72
分支2	85.53	82.18	70.53