JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE) ›› 2016, Vol. 46 ›› Issue (3): 7-13.doi: 10.6040/j.issn.1672-3961.2.2015.106

Previous Articles     Next Articles

A new multi-focus image fusion method based on deep neural network model

LIU Fan, CHEN Zehua, CHAI Jing   

  1. College of Information Engineering, Taiyuan University of Technology, Taiyuan 030024, Shanxi, China
  • Received:2015-06-23 Online:2016-06-30 Published:2015-06-23

Abstract: There existed low-frequency information distortion phenomenon in fusing multi-focus images. Aimed to solve the problem, a new fusion strategy based on deep neural network model was proposed for fusing low-frequency subbands. Combined with Wavelet Kernel Filter and traditional fusion strategy for high-frequency subbands, a new fusion method for fusing multi-focus images was given. The method extracted efficient features by using AutoEncoder model. The experimental results showed that proposed method could obtain better images. The edge fusion qualify value of the proposed fusion result was 0.802 7, compared with traditional fusion strategy, contourlet-based multi-focus method and non-sampled contourlet-based multi-focus method, 0.761 4, 0.722 7, and 0.716 4, which could provide an effective method for fusing multi-focus images.

Key words: autoencoder, deep neural network, image fusion, multi-focus image, wavelet kernel filter

CLC Number: 

  • TP183
[1] POHL C, VAN GENDEREN J L. Multisensor image fusion in remote sensing: concepts, methods and applications[J]. International Journal of Remote Sensing, 1998, 19(5):823-854.
[2] BURT P J, ADELSON E H. The Laplacian pyramid as a compact image code[J]. IEEE Transactions on Communication, 1983, 31(4): 532-540.
[3] ZAVORIN I, LE MOIGNE J. Use of multiresolution wavelet feature pyramids for automatic registration of multisensor imagery[J]. IEEE Transactions on Image Processing, 2005, 14(6): 770-782.
[4] REDONDO R, ŠROUBEK F, FISCHER S, et al. Multifocus image fusion using the log-Gabor transform and a multisize windows technique[J]. Information Fusion, 2009, 10(2):163-171.
[5] BORWONWATANADELOK P, RATTANAPITAK W, UDOMHUNSAKUL S. Multi-focus image fusion based on stationary wavelet transform and extended spatial frequency measurement[C] //IEEE International Conference of Electronic Computer Technology. Macau, China: IEEE Press, 2009: 77-81.
[6] DO M N, VETTERLI M. The Contourlet transform: an efficient directional multiresolution image representation[J]. IEEE Transactions on Image Processing, 2005, 14(12): 2091-2106.
[7] YANG S Y, WANG M, JIAO L C. Image fusion based on a new contourlet packet[J]. Information Fusion, 2010, 11(2): 78-84.
[8] DA CUNHA A L, ZHOU J P, DO M N. The nonsubsampled contourlet transform: theory, design, and applications[J]. IEEE Transactions on Image Processing, 2006, 15(10):3089-3101.
[9] ZHENG S, SHI W Z, LIU J, et al. Multisource image fusion method using support value transform[J]. IEEE Transactions on Image Processing, 2007, 16(7):1831-1839.
[10] YANG B, LI S T. Multifocus image fusion and restoration with sparse representation[J]. IEEE Transactions on Instrumentation and Measurement, 2010, 59(4): 884-892.
[11] PETROVIC V. Subjective tests for image fusion evaluation and objective metric validation[J]. Information Fusion, 2007, 8(2): 208-216.
[12] BENGIO Y. Learning deep architectures for AI[J]. Journal Foundations and Trends® in Machine Learning, 2009, 2(1): 1-127.
[13] HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural computation, 2006, 18(7):1527-1554.
[14] BENGIO Y, LAMBLIN P, POPOVICI D, et al. Greedy layer-wise training of deep networks[C] //Advances in Neural Information Processing Systems 19(NIPS'06). Vancouver, Canada: MIT Press, 2007: 153-160.
[15] HINDON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
[16] COATES A, NG A Y, LEE H. An analysis of single-layer networks in unsupervised feature learning[M]. Brookline: Microtome Publishing, 2011: 215-223.
[17] ZHENG W L, ZHU J Y, PENG Y, et al. EEG-Based emotion classification using deep belief networks[C] //IEEE International Conference on Multimedia and Expo. Chengdu, China: IEEE press, 2014: 1-6.
[18] TANG J X, DENG C W, HUANG G B, et al. Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(3): 1174-1185.
[19] HUANG P H, HUANG Y, WANG W, et al. Deep embedding network for clustering[C] //22nd International Conference on Pattern Recognition. Stockholm, Sweden: IEEE press, 2014: 1532-1537.
[20] LIU F, JIAO L C, YANG S Y. SAR image despeckling based on wavelet kernel transform and gaussian scale mixture model[C] //2nd Asian-Pacific Conference on Synthetic Aperture Radar. Xi'an, China: IEEE Press, 2009: 1088-1091.
[21] HUANG X S, CHEN Z. A wavelet-based image fusion algorithm[C] //IEEE Region 10 Conference on Computers, Communications, Control and Power Engineering. Beijing, China: IEEE Press, 2002: 602-605.
[22] 王海晖, 彭嘉雄, 吴巍, 基于小波包变换的遥感图像融合[J]. 中国图像图形学报, 2002, 7(9): 922-937. WANG Haihui, PENG Jiaxiong, WU Wei. Remote sensing image fusion using wavelet packet transform[J]. Journal of Image and Graphics, 2002, 7(9):922-937.
[23] 刘贵喜, 杨万海. 基于小波分解的图像融合方法及性能评价[J]. 自动化学报, 2002, 28(6):927-934. LIU Guixi, YANG Wanhai. A Wavelet decomposition-based image fusion scheme and its Performance Evaluation[J]. ACTA Automatica Sinica, 2002, 28(6):927-934.
[24] WANG Z, BOVIK A C. A universal image quality index[J]. IEEE Signal Processing Letters, 2002, 9(3):81-84.
[25] PIELLA G, HEIJMANS H. A new quality metric for image fusion[C] //International Conference on Image Processing. Barcelona, Spain: IEEE Press, 2003:III-173-6 vol.2.
[1] Dongdong SHEN,Fengyu ZHOU,Mengyuan LI,Shuqian WANG,Renhe GUO. Indoor wireless positioning based on ensemble deep neural network [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 95-102.
[2] Haiyong CHEN,Li YU,Hui LIU,Jiabo YANG,Qidi HU. Solar cell defect images fusion based on empirical wavelet [J]. Journal of Shandong University(Engineering Science), 2018, 48(5): 24-31.
[3] TANG Leshuang, TIAN Guohui, HUANG Bin. An object fusion recognition algorithm based on DSmT [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(1): 50-56.
[4] YANG Xiulin1, HUANG Shuo2*, DENG Miao1, ZHANG Jihong1,3. Image fusion method based on saliency computation and adaptive PCNN [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2014, 44(2): 35-42.
[5] LIN Zhe1, YAN Jing-wen2, YUAN Ye2. Multi-modality image fusion based on sparse representation and PCNN [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2013, 43(4): 13-17.
[6] WU Guo-yao1, MA Li-yong2. A method based on FFD B-spline registration of the iris image fusion [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2010, 40(5): 24-27.
[7] CHEN Shuang,JIANG Wei . Medical image fusion based on DT-DWT [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2008, 38(1): 32-35 .
Full text



[1] CHENG Daizhan, LI Zhiqiang. A survey on linearization of nonlinear systems[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 26 -36 .
[2] WANG Yong, XIE Yudong. Gas control technology of largeflow pipe[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 70 -74 .
[3] LIU Xin 1, SONG Sili 1, WANG Xinhong 2. [J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 98 -100 .
[5] CHEN Huaxin, CHEN Shuanfa, WANG Binggang. The aging behavior and mechanism of base asphalts[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(2): 125 -130 .
[7] LI Shijin, WANG Shengte, HUANG Leping. Change detection with remote sensing images based on forward-backward heterogenicity[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2018, 48(3): 1 -9 .
[8] ZHAO Ke-Jun, WANG Xin-Jun, LIU Xiang, CHOU Yi-Hong. Algorithms of continuous top-k join query over structured overlay networks[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2009, 39(5): 32 -37 .
[9] ZHAO Zhi-guang,WANG Deng-jie,TIAN Yun-fei . Roadbed settlement based on the gray theory[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(3): 86 -88 .
[10] YAO Zhan-yong,SHANG Qing-sen,ZHAO Zhi-zhong,JIA Zhao-xia . The influence analysis of the semirigid asphalt pavement configuration stress and distortion by interface conditions[J]. JOURNAL OF SHANDONG UNIVERSITY (ENGINEERING SCIENCE), 2007, 37(3): 93 -99 .