一种使用并行交错采样进行超分辨的方法

doi:10.6040/j.issn.1672-3961.0.2019.318

摘要/Abstract

摘要：

目前基于互联网各类图像以及人工智能应用对图像数据质量较为敏感,但由于采集设备以及传输方式限制,导致图像质量受到非常严重影响,为了弥补图像数据质量损失和增强图像效果,提出一种并行交错上下采样网络(parallel interlaced up and down sampling network, PSUDN)作为一个解决该问题的更好的方案,利用并行的高分辨特征(high resolution feature, HR_Feature)和低分辨特征(low resolution feature, LR_Feature),交错进行采样生成高级特征图,通过构建并行的高分辨率特征模块和低分辨率特征模块提升输出的高分辨率图片的质量。通过并行的上采样和下采样,构建的模型可以重建8倍的高分辨率图片,并达到当前较好的效果。

关键词: 超分辨, 残差网络, 并行采样网络, 图像重建

Abstract:

Various Internet-based images and artificial intelligence applications were more sensitive to the quality of image data. The image quality had been seriously affected due to the limitations of previous acquisition equipment and transmission methods. In order to compensate for the loss of image data quality and enhance the image effect, a parallel interlaced up and down sampling network (PSUDN) was proposed as a better solution to this problem, which using parallel high resolution feature (HR Feature) and low resolution feature (LR Feature) interleaving sample to generated advanced feature maps, and improved the quality of the output high-resolution pictures by building parallel high resolution feature modules and low resolution feature modules. The model constructed by parallel upsampling and downsampling could reconstruct 8 times high resolution pictures and achieved better results.

Key words: DCNN, super resolution, residual network, parallel interlaced, rebuild high quality high resolution images

中图分类号:

TP319

朱安, 徐初. 一种使用并行交错采样进行超分辨的方法[J]. 山东大学学报 (工学版), 2020, 50(2): 10-16.

An ZHU, Chu XU. Method for super-resolution using parallel interlaced sampling[J]. Journal of Shandong University(Engineering Science), 2020, 50(2): 10-16.

图/表 5

图1

图2

表1

图3

表2

参考文献 30

1	AGHAJAN H K , KAILATH T . Sensor array processing techniques for super resolution multi-line-fitting and straight edge detection[J]. IEEE Transactions on Image Processing a Publication of the IEEE Signal Processing Society, 1993, 2 (4): 454- 65. doi: 10.1109/83.242355
2	GERCHBERG R . Super-resolution through error energy reduction[J]. J Modern Optics, 1974, 21 (9): 709- 720.
3	GLASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//2009 IEEE 12th International Conference on Computer Vision (ICCV). Maimi, USA: IEEE, 2009.
4	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2015.
5	KAPPELER A , YOO S , DAI Q , et al. Video super-resolution with convolutional neural networks[J]. IEEE Transactions on Computational Imaging, 2016, 2 (2): 109- 122. doi: 10.1109/TCI.2016.2532323
6	MUHAMMAD H , RAHMAT W M , HAJIME N . Inception learning super-resolution[J]. Applied Optics, 2017, 56 (22): 6043- 6053. doi: 10.1364/AO.56.006043
7	SHI W, CABALLERO J, HUSZAR, FERENC, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016.
8	LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017.
9	TAI Y, YANG J, LIU X. Image super-resolution via deep recursive residual network[C]// IEEE Computer Vision and Pattern Recognition (CVPR 2017). Honolulu, USA: IEEE, 2017.
10	DONG C , LOY C C , HE K , et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell, 2014, 38 (2): 295- 307.
11	KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2015.
12	KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2015.
13	DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//ECCV2016 14th European Conference. Amsterdam, the Netherlands: ECCV, 2016.
14	LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Honolulu, USA: IEEE, 2017.
15	MAO X J, SHEN C, YANG Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]//Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems. Barcelona, Spain: NIPS, 2016.
16	BLAU Yochai, MICHAELI Tomer. The perception-distortion tradeoff[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake, USA: IEEE, 2018.
17	TAI Y, YANG J, LIU X. Image super-resolution via deep recursive residual network[C]//IEEE Computer Vision and Pattern Recognition (CVPR 2017). Honolulu, USA: IEEE, 2017.
18	LIU P, ZHANG H, ZHANG K, et al. Multi-level wavelet-CNN for image restoration[C]//IEEE 2018 Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE, 2018.
19	IRANI M , PELEG S . Motion analysis for image enhancement: resolution, occlusion, and transparency[J]. Journal of Visual Communication and Image Representation, 1993, 4 (4): 324- 335. doi: 10.1006/jvci.1993.1030
20	WANG Z , BOVIK A C , SHEIKH H R , et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612. doi: 10.1109/TIP.2003.819861
21	DENG Ruoxi , LIU Shengjun . Relative depth order estimation using multi-scale densely connected convolutional networks densely connected convolutional networks[J]. IEEE, 2019, 7 (1): 38630- 38643.
22	SZEGEDY C , LIU W , JIA Y , et al. Going deeper with convolutions[J]. CoRR, 2014, 2 (1): 1409- 1420.
23	AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: dataset and study[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, USA: IEEE, 2017.
24	KIM K I , KWON Y . Single-image super-resolution using sparse regression and natural image prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (6): 1127- 1133. doi: 10.1109/TPAMI.2010.25
25	ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]//Curves and Surfaces 7th International Conference, 2010. Avignon, France: Springer, 2012.
26	MARTIN D R , FOWLKES C , TAL D , et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[J]. Proceedings of the Eighth International Conference On Computer Vision, 2001, 2 (11): 416- 423.
27	HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015.
28	KANNAN A, YOUNG P, RAMAVAJJALA V, et al. Smart reply: automated response suggestion for email[C]//22nd ACM SIGKDD International Conference. San Francisco, USA: ACM, 2016.
29	RAMAKRISHNAN B , RAO S S . A general loss function based optimization procedure for robust design[J]. Engineering Optimization, 1996, 25 (4): 255- 276. doi: 10.1080/03052159608941266
30	ALZATE C , SUYKENS J Kernel . Component analysis using an epsilon-insensitive robust loss function[J]. IEEE Transactions on Neural Networks, 2008, 19 (9): 1583- 1598. doi: 10.1109/TNN.2008.2000443

多维度评价

Viewed

Full text

Abstract

Cited

Shared

Discussed

算法	比例	SET5		SET14		BSDS100		URBAN100		MANGA109
算法	比例	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM	PSNR	SSIM
Bicubic	2	33.650	0.930	30.340	0.870	29.560	0.844	26.880	0.841	30.840	0.935
VDSR	2	37.530	0.958	32.970	0.913	31.900	0.896	30.770	0.914	37.160	0.974
DRCN	2	37.630	0.959	32.980	0.913	31.850	0.894	30.760	0.913	37.570	0.973
LapSRN	2	37.520	0.959	33.080	0.922	31.800	0.895	30.410	0.910	37.270	0.974
EDSR	2	38.110	0.960	33.920	0.919	33.320	0.901	32.930	0.935	39.100	0.977
PSUDN(Ours)	2	38.130	0.961	33.950	0.919	23.370	0.910	32.900	0.928	39.420	0.977
Bicubic	4	28.420	0.810	26.100	0.704	25.960	0.669	23.150	0.659	24.920	0.789
VDSR	4	31.350	0.882	28.030	0.770	27.290	0.726	25.180	0.753	28.820	0.886
DRCN	4	31.530	0.884	28.040	0.770	27.240	0.724	25.140	0.752	28.970	0.886
LapSRN	4	31.540	0.885	28.190	0.772	27.320	0.728	25.210	0.756	29.090	0.890
EDSR	4	32.460	0.897	28.800	0.788	27.710	0.742	26.630	0.803	31.020	0.915
PSUDN(Ours)	4	32.490	0.899	28.810	0.791	27.710	0.742	26.800	0.810	31.510	0.917
Bicubic	8	24.390	0.657	23.190	0.568	23.670	0.547	20.740	0.515	21.470	0.649
VDSR	8	25.720	0.711	24.210	0.609	24.370	0.576	21.540	0.560	22.830	0.707
LapSRN	8	26.140	0.738	24.440	0.623	24.540	0.586	21.810	0.581	23.390	0.735
EDSR	8	26.970	0.775	24.940	0.640	24.800	0.596	22.470	0.620	24.580	0.778
PSUDN(Ours)	8	27.150	0.779	25.010	0.643	24.880	0.601	22.450	0.622	25.030	0.781

数据集	Loss	PSNR	SSIM	Training epochs
DIV2K	L₁	26.680	0.714	10 000
DIV2K	L₂	26.200	0.703	10 000
DIV2K	Charbinnier	26.830	0.719	10 000

[1]	黄彩云,陈德武,何吉福,胡艺,王楠,陈沛. 基于改进双路径网络的上肢肌肉骨骼异常检测[J]. 山东大学学报 (工学版), 2022, 52(3): 25-33.
[2]	蒋桐雨, 陈帆, 和红杰. 基于非对称U型金字塔重建的轻量级人脸超分辨率网络[J]. 山东大学学报 (工学版), 2022, 52(1): 1-8.
[3]	张月芳,邓红霞,呼春香,钱冠宇,李海芳. 融合残差块注意力机制和生成对抗网络的海马体分割[J]. 山东大学学报 (工学版), 2020, 50(6): 76-81.
[4]	黄劲潮. 深度残差特征与熵能量优化运动目标跟踪算法[J]. 山东大学学报 (工学版), 2019, 49(4): 14-23.
[5]	颜子夜,陆耀,李建武,马跃. 一种基于核主成分分析的图像超分辨率算法[J]. 山东大学学报(工学版), 2011, 41(4): 101-105.
[6]	浦剑1 ,张军平1 ，黄华2 . 超分辨率算法研究综述[J]. 山东大学学报(工学版), 2009, 39(1): 27-32.