Journal of Shandong University(Engineering Science) ›› 2020, Vol. 50 ›› Issue (2): 10-16.doi: 10.6040/j.issn.1672-3961.0.2019.318

• Machine Learning & Data Mining • Previous Articles     Next Articles

Method for super-resolution using parallel interlaced sampling

An ZHU1(),Chu XU2   

  1. 1. Software Institute, Nanjing University, Nanjing 210000, Jiangsu, China
    2. Information Construction Management Service Center, Nanjing University Information Construction Management Service Center, Nanjing 210000, Jiangsu, China
  • Received:2019-06-20 Online:2020-04-20 Published:2020-04-16

RichHTML

15

PDF (PC)

331

Abstract:

Various Internet-based images and artificial intelligence applications were more sensitive to the quality of image data. The image quality had been seriously affected due to the limitations of previous acquisition equipment and transmission methods. In order to compensate for the loss of image data quality and enhance the image effect, a parallel interlaced up and down sampling network (PSUDN) was proposed as a better solution to this problem, which using parallel high resolution feature (HR Feature) and low resolution feature (LR Feature) interleaving sample to generated advanced feature maps, and improved the quality of the output high-resolution pictures by building parallel high resolution feature modules and low resolution feature modules. The model constructed by parallel upsampling and downsampling could reconstruct 8 times high resolution pictures and achieved better results.

Key words: DCNN, super resolution, residual network, parallel interlaced, rebuild high quality high resolution images

CLC Number: 

  • TP319

Fig.1

The network of PSUDN"

Fig.2

The unit for parallel interlaced up and down sampling"

Table 1

Quantitative evaluation of algorithm performance [scale factors 2x, 4x and 8x]"

算法 比例 SET5 SET14 BSDS100 URBAN100 MANGA109
PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM PSNR SSIM
Bicubic 2 33.650 0.930 30.340 0.870 29.560 0.844 26.880 0.841 30.840 0.935
VDSR 2 37.530 0.958 32.970 0.913 31.900 0.896 30.770 0.914 37.160 0.974
DRCN 2 37.630 0.959 32.980 0.913 31.850 0.894 30.760 0.913 37.570 0.973
LapSRN 2 37.520 0.959 33.080 0.922 31.800 0.895 30.410 0.910 37.270 0.974
EDSR 2 38.110 0.960 33.920 0.919 33.320 0.901 32.930 0.935 39.100 0.977
PSUDN(Ours) 2 38.130 0.961 33.950 0.919 23.370 0.910 32.900 0.928 39.420 0.977
Bicubic 4 28.420 0.810 26.100 0.704 25.960 0.669 23.150 0.659 24.920 0.789
VDSR 4 31.350 0.882 28.030 0.770 27.290 0.726 25.180 0.753 28.820 0.886
DRCN 4 31.530 0.884 28.040 0.770 27.240 0.724 25.140 0.752 28.970 0.886
LapSRN 4 31.540 0.885 28.190 0.772 27.320 0.728 25.210 0.756 29.090 0.890
EDSR 4 32.460 0.897 28.800 0.788 27.710 0.742 26.630 0.803 31.020 0.915
PSUDN(Ours) 4 32.490 0.899 28.810 0.791 27.710 0.742 26.800 0.810 31.510 0.917
Bicubic 8 24.390 0.657 23.190 0.568 23.670 0.547 20.740 0.515 21.470 0.649
VDSR 8 25.720 0.711 24.210 0.609 24.370 0.576 21.540 0.560 22.830 0.707
LapSRN 8 26.140 0.738 24.440 0.623 24.540 0.586 21.810 0.581 23.390 0.735
EDSR 8 26.970 0.775 24.940 0.640 24.800 0.596 22.470 0.620 24.580 0.778
PSUDN(Ours) 8 27.150 0.779 25.010 0.643 24.880 0.601 22.450 0.622 25.030 0.781

Fig.3

Visualization of the result [4x upscaling]"

Table 2

Quantitative comparison of loss function"

数据集 Loss PSNR SSIM Training epochs
DIV2K L1 26.680 0.714 10 000
DIV2K L2 26.200 0.703 10 000
DIV2K Charbinnier 26.830 0.719 10 000
1 AGHAJAN H K , KAILATH T . Sensor array processing techniques for super resolution multi-line-fitting and straight edge detection[J]. IEEE Transactions on Image Processing a Publication of the IEEE Signal Processing Society, 1993, 2 (4): 454- 65.
doi: 10.1109/83.242355
2 GERCHBERG R . Super-resolution through error energy reduction[J]. J Modern Optics, 1974, 21 (9): 709- 720.
3 GLASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//2009 IEEE 12th International Conference on Computer Vision (ICCV). Maimi, USA: IEEE, 2009.
4 HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Conference on Computer Vision and Pattern Recognition. New York, USA: IEEE, 2015.
5 KAPPELER A , YOO S , DAI Q , et al. Video super-resolution with convolutional neural networks[J]. IEEE Transactions on Computational Imaging, 2016, 2 (2): 109- 122.
doi: 10.1109/TCI.2016.2532323
6 MUHAMMAD H , RAHMAT W M , HAJIME N . Inception learning super-resolution[J]. Applied Optics, 2017, 56 (22): 6043- 6053.
doi: 10.1364/AO.56.006043
7 SHI W, CABALLERO J, HUSZAR, FERENC, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016.
8 LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017.
9 TAI Y, YANG J, LIU X. Image super-resolution via deep recursive residual network[C]// IEEE Computer Vision and Pattern Recognition (CVPR 2017). Honolulu, USA: IEEE, 2017.
10 DONG C , LOY C C , HE K , et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell, 2014, 38 (2): 295- 307.
11 KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2015.
12 KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution[C]//Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2015.
13 DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//ECCV2016 14th European Conference. Amsterdam, the Netherlands: ECCV, 2016.
14 LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). Honolulu, USA: IEEE, 2017.
15 MAO X J, SHEN C, YANG Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]//Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems. Barcelona, Spain: NIPS, 2016.
16 BLAU Yochai, MICHAELI Tomer. The perception-distortion tradeoff[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake, USA: IEEE, 2018.
17 TAI Y, YANG J, LIU X. Image super-resolution via deep recursive residual network[C]//IEEE Computer Vision and Pattern Recognition (CVPR 2017). Honolulu, USA: IEEE, 2017.
18 LIU P, ZHANG H, ZHANG K, et al. Multi-level wavelet-CNN for image restoration[C]//IEEE 2018 Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE, 2018.
19 IRANI M , PELEG S . Motion analysis for image enhancement: resolution, occlusion, and transparency[J]. Journal of Visual Communication and Image Representation, 1993, 4 (4): 324- 335.
doi: 10.1006/jvci.1993.1030
20 WANG Z , BOVIK A C , SHEIKH H R , et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13 (4): 600- 612.
doi: 10.1109/TIP.2003.819861
21 DENG Ruoxi , LIU Shengjun . Relative depth order estimation using multi-scale densely connected convolutional networks densely connected convolutional networks[J]. IEEE, 2019, 7 (1): 38630- 38643.
22 SZEGEDY C , LIU W , JIA Y , et al. Going deeper with convolutions[J]. CoRR, 2014, 2 (1): 1409- 1420.
23 AGUSTSSON E, TIMOFTE R. NTIRE 2017 challenge on single image super-resolution: dataset and study[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Honolulu, USA: IEEE, 2017.
24 KIM K I , KWON Y . Single-image super-resolution using sparse regression and natural image prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32 (6): 1127- 1133.
doi: 10.1109/TPAMI.2010.25
25 ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]//Curves and Surfaces 7th International Conference, 2010. Avignon, France: Springer, 2012.
26 MARTIN D R , FOWLKES C , TAL D , et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[J]. Proceedings of the Eighth International Conference On Computer Vision, 2001, 2 (11): 416- 423.
27 HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars[C]//IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015.
28 KANNAN A, YOUNG P, RAMAVAJJALA V, et al. Smart reply: automated response suggestion for email[C]//22nd ACM SIGKDD International Conference. San Francisco, USA: ACM, 2016.
29 RAMAKRISHNAN B , RAO S S . A general loss function based optimization procedure for robust design[J]. Engineering Optimization, 1996, 25 (4): 255- 276.
doi: 10.1080/03052159608941266
30 ALZATE C , SUYKENS J Kernel . Component analysis using an epsilon-insensitive robust loss function[J]. IEEE Transactions on Neural Networks, 2008, 19 (9): 1583- 1598.
doi: 10.1109/TNN.2008.2000443
[1] ZHANG Yuefang, DENG Hongxia, HU Chunxiang, QIAN Guanyu, LI Haifang. Hippocampal segmentation combining residual attention mechanism and generative adversarial networks [J]. Journal of Shandong University(Engineering Science), 2020, 50(6): 76-81.
[2] Jinchao HUANG. Object tracking algorithm based on deep residual features and entropy energy optimization [J]. Journal of Shandong University(Engineering Science), 2019, 49(4): 14-23.
[3] YAN Zi-ye, LU Yao, LI Jian-wu, MA Yue. Kernel principal components analysis based super resolution method [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2011, 41(4): 101-105.
[4] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(1): 27-32.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] CHEN Rui, LI Hongwei, TIAN Jing. The relationship between the number of magnetic poles and the bearing capacity of radial magnetic bearing[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(2): 81 -85 .
[2] HAO Ranhang,CHEN Shouyu . The theory, model and method of water resources evaluationombining quantity with quality[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2006, 36(3): 46 -50 .
[3] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 104 -107 .
[4] . [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 131 -136 .
[5] SONG Qing,LI Xiao-lei,ZHANG Cheng-jin . Optimization of a postal express mail network based on bottleneck analysis[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(5): 29 -33 .
[6] HU Hong-chun,WU Yao-hua,LIAO Li . Routing optimization for logistics distribution and its application[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(4): 104 -107 .
[7] XUE Hongtao,TIAN Guohui,LI Xiaolei,LU Fei . Application of the QR Code for various object identificationand manipulation[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(6): 25 -30 .
[8] MA Qi-Hua, WANG Yi-Tai. Application of the high density resistivity method to surrvey huge empty water  outside of a coal mine[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(4): 107 -111 .
[9] GAO Ming, SHI Yue-Thao, WANG Ni-Ni, SUN Feng-Zhong, PING Ya-Ming. Circumferential inflow air distributing rules in a natural draft  wet-cooling tower under crosswind conditions[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(3): 154 -158 .
[10] LIU Wen-liang, ZHU Wei-hong, CHEN Di, ZHANG Hong-quan. Detection and tracking of moving targets using the morphology match in radar images[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(3): 31 -36 .
Copyright 2018 © Editorial office of Journal of Shandong University (Engineering Science)
Supported by Beijing Magtech Co.Ltd