Computer Science ›› 2018, Vol. 45 ›› Issue (6A): 162-166.

• Pattern Recognition & Image Processing • Previous Articles     Next Articles

Infrared and Visible Images Fusion Using Visual Saliency and Dual-PCNN

HOU Rui-chao,ZHOU Dong-ming,NIE Ren-can,LIU Dong,GUO Xiao-peng   

  1. School of Information Science & Engineering,Yunnan University,Kunming 650504,China
  • Online:2018-06-20 Published:2018-08-03

Abstract: Aiming at uneven brightness,inconspicuous object,low contrast and loss details problems in the existing infrared and visible light image fusion methods,in combination with nonsubsampled shearlet transform (NSST) which has multi-scale transformation and the most sparse expression characteristics,saliency detection which has the advantage of highlighting infrared objects,and Dual-channel pulse coupled neural network(Dual-PCNN)which has the advantages of coupling and pulse synchronization,an image fusion method for infrared and visible light images based on NSST and visual saliency guide Dual-PCNN was proposed in this paper.Firstly,the high frequency and low frequency sub-band coefficients of infrared and visible light image are decomposed by NSST in each direction,and then low frequency coefficients are fused by the Dual-PCNN,which is guided by the saliency map of the images.For the high frequency sub-band coefficients,a modified spatial frequency is adopted as the input to motivate the Dual-PCNN.Finally,the fused image is reconstructed by inverse NSST.The experimental results demonstrate that the infrared objects in the fusion image are highlighted and the details of the visible background are rich.Compared with other fusion algorithms,the proposed method has a certain degree of improvement on the subjective evaluation and objective evaluation.

Key words: Dual-channel pulse coupled neural network, Image fusion, Nonsubsampled shearlet transform, Visual saliency

CLC Number: 

  • TP391.41
[1]JIN X,JIANG Q,YAO S,et al.A survey of infrared and visual image fusion methods[J].Infrared Physics & Technology,2017,85:478-501.
[2]LIU S,CHEN J.A fast multi-focus image fusion algorithm by DWT and focused region decision map[C]∥Signal and Information Processing Association Summit and Conference.IEEE,2017:1-7.
[3]DO M N,VETTERLI M.The contourlet transform:an efficient directional multiresolution image representation [J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2005,14(12):2091.
[4]EASLEY G,LABATE D,LIM W Q.Sparse directional image representations using the discrete shearlet transform[J].Applied & Computational Harmonic Analysis,2008,25(1):25-46.
[5]DA C A,ZHOU J,DO M N.The nonsubsampled contourlet transform:theory,design,and applications [J].IEEE Transactions on Image Processing,2006,15(10):3089-3101.
[6]HOU B,ZHANG X,BU X,et al.SAR image despeckling based on nonsubsampled shearlet transform [J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sen-sing,2012,5(3):809-823.
[7]LIU H,ZHU T,ZHAO J.Infrared and visible image fusion based on region of interest detection and nonsubsampled con-tourlet transform [J].Journal of Shanghai Jiaotong University,2013,18(5):526-534.
[8]ZHAO J,ZHOU Q,CHEN Y,et al.Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition [J].Infrared Physics & Technology,2013,56(2):93-99.
[9]FU Z Z,WANG X,LI X F.Infrared and visible image fusion based on visual saliency and NSCT [J].Journal of University of Electronic Science and Technology of China,2017,46(2):357-362.
[10]ECKHORN R,REITBOECK H J,ARNDT M,et al.Feature linking via synchronization among distributed assemblies:Simulations of results from cat visual cortex [J].Neural Computation,1990,2(3):293-307.
[11]KONG W,ZHANG L,LEI Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN [J].Infrared Physics & Technology,2014,65(7):103-112.
[12]BAKIC' D,KRISHTAL I,WILSON E N.Parseval frame wavelets with En(2)-dilations [J].Applied & Computational Harmonic Analysis,2005,19(3):386-431.
[13]HUANG Z,DING M,ZHANG X.Medical image fusion based on non-subsampled shearlet transform and spiking cortical mo-del [J].Journal of Medical Imaging & Health Informatics,2017.
[14]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency- tuned salient region detection[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2009(CVPR 2009).IEEE,2010:1597-1604.
[15]HE K,SUN J,TANG X.Guided image filtering [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(6):1397.
[16]ITTI L,KOCH C,NIEBUR E.A Model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1998,20(11):1254-1259.
[17]ZHANG L,TONG M H,MARKS T K,et al.SUN:A bayesian framework for saliency using natural statistics [J].Journal of Vision,2008,8(7):1-20.
[18]HOU X,ZHANG L.Saliency Detection:A spectral residual approach[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2007(CVPR’07).IEEE,2007:1-8.
[19]JIA Y,RONG C,ZHU Y,et al.Multi-focus Image Fusion Sch- eme Using Adaptive Dual-Channel Pulse Coupled Neural Network[C]∥International Conference on Intelligent Human-Machine Systems and Cybernetics.IEEE,2016:147-151.
[20]HUANG F,ZHOU D,NIE R,et al.A multi-focus color image fusion algorithm based on an adaptive SF-PCNN in NSCT domain[C]∥International Conference on Digital Image Proces-sing.2017:1042021.
[21]WANG W,CHANG F.A Multi-focus Image Fusion Method Based on Laplacian Pyramid[J].Journal of Computers,2011,6(12):2559-2566.
[22]WANG X,YAO L,SONG R,et al.A New Infrared and Visible Image Fusion Algorithm in NSCT Domain[C]∥International Conference on Intelligent Computing.Springer,Cham,2017:420-431.
[23]YANG B,LI S.Pixel-level image fusion with simultaneous orthogonal matching pursuit [J].Information Fusion,2012,13(1):10-19
[1] ZHAO Ming-hua, ZHOU Tong-tong, DU Shuang-li, SHI Zheng-hao. Single Backlit Image Enhancement Based on Virtual Exposure Method [J]. Computer Science, 2022, 49(6A): 384-389.
[2] LAI Teng-fei, ZHOU Hai-yang, YU Fei-hong. Real-time Extend Depth of Field Algorithm for Video Processing [J]. Computer Science, 2022, 49(6A): 314-318.
[3] GAO Yuan-hao, LUO Xiao-qing, ZHANG Zhan-cheng. Infrared and Visible Image Fusion Based on Feature Separation [J]. Computer Science, 2022, 49(5): 58-63.
[4] YAN Min, LUO Xiao-qing, ZHANG Zhan-cheng. Infrared and Visible Image Fusion Network Based on Optical Transmission Model Learning [J]. Computer Science, 2022, 49(4): 215-220.
[5] GUAN Zheng, DENG Yang-lin, NIE Ren-can. Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion [J]. Computer Science, 2021, 48(9): 153-159.
[6] HUANG Xiao-sheng, XU Jing. Multi-focus Image Fusion Method Based on PCANet in NSST Domain [J]. Computer Science, 2021, 48(9): 181-186.
[7] TIAN Song-wang, LIN Su-zhen, YANG Bo. Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator [J]. Computer Science, 2021, 48(8): 185-190.
[8] WANG Li-fang, WANG Rui-fang, LIN Su-zhen, QIN Pin-le, GAO Yuan, ZHANG Jin. Multimodal Medical Image Fusion Based on Dual Residual Hyper Densely Networks [J]. Computer Science, 2021, 48(2): 160-166.
[9] ZHU Zhen, HUANG Rui, ZANG Tie-gang, LU Shi-jun. Single Image Defogging Method Based on Weighted Near-InFrared Image Fusion [J]. Computer Science, 2020, 47(8): 241-244.
[10] YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection [J]. Computer Science, 2020, 47(7): 84-91.
[11] LIU Jun-qi,LI Zhi,ZHANG Xue-yang. Review of Maritime Target Detection in Visible Bands of Optical Remote Sensing Images [J]. Computer Science, 2020, 47(3): 116-123.
[12] ZHU Ying,XIA Yi-li,PEI Wen-jiang. Fusion of Infrared and Color Visible Images Based on Improved BEMD [J]. Computer Science, 2020, 47(3): 124-129.
[13] LI Chang-xing, LEI Liu, ZHANG Xiao-lu. Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN [J]. Computer Science, 2020, 47(10): 194-199.
[14] LI Xiao-yu,GAO Qing-wei,LU Yi-xiang,SUN Dong. Image Fusion Method Based on Image Energy Adjustment [J]. Computer Science, 2020, 47(1): 153-158.
[15] ZHANG Bing, XIE Cong-hua, LIU Zhe. Multi-focus Image Fusion Based on Latent Sparse Representation and Neighborhood Information [J]. Computer Science, 2019, 46(9): 254-258.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!