结合视觉显著性与Dual-PCNN的红外与可见光图像融合

摘要/Abstract

摘要： 针对现存的红外与可见光图像融合算法亮度不均、目标不突出、对比度不高、细节丢失等问题,结合非下采样剪切波变换(NSST)具有多尺度、最具稀疏表达的特性,显著性检测具有突出红外目标的优势,双通道脉冲耦合神经网络(Dual-PCNN)具有耦合、脉冲同步激发等优点,提出一种基于NSST结合视觉显著性引导Dual-PCNN的图像融合方法。首先,通过NSST分解红外与可见光图像各方向的高频与低频子带系数;然后,低频子带系数采用基于显著性决策图引导Dual-PCNN融合策略,高频子带系数采用改进的空间频率作为优化Dual-PCNN的激励进行融合;最后,经过NSST逆变换得到融合图像。实验结果表明,融合图像红外目标突出且可见光背景细节丰富。该方法相比于其他融合算法在主观评价与客观评价上都有一定程度的改善。

关键词: 非下采样剪切波变换, 视觉显著性, 双通道脉冲耦合神经网络, 图像融合

Abstract: Aiming at uneven brightness,inconspicuous object,low contrast and loss details problems in the existing infrared and visible light image fusion methods,in combination with nonsubsampled shearlet transform (NSST) which has multi-scale transformation and the most sparse expression characteristics,saliency detection which has the advantage of highlighting infrared objects,and Dual-channel pulse coupled neural network(Dual-PCNN)which has the advantages of coupling and pulse synchronization,an image fusion method for infrared and visible light images based on NSST and visual saliency guide Dual-PCNN was proposed in this paper.Firstly,the high frequency and low frequency sub-band coefficients of infrared and visible light image are decomposed by NSST in each direction,and then low frequency coefficients are fused by the Dual-PCNN,which is guided by the saliency map of the images.For the high frequency sub-band coefficients,a modified spatial frequency is adopted as the input to motivate the Dual-PCNN.Finally,the fused image is reconstructed by inverse NSST.The experimental results demonstrate that the infrared objects in the fusion image are highlighted and the details of the visible background are rich.Compared with other fusion algorithms,the proposed method has a certain degree of improvement on the subjective evaluation and objective evaluation.

Key words: Dual-channel pulse coupled neural network, Image fusion, Nonsubsampled shearlet transform, Visual saliency

中图分类号:

TP391.41

侯瑞超,周冬明,聂仁灿,刘栋,郭晓鹏. 结合视觉显著性与Dual-PCNN的红外与可见光图像融合[J]. 计算机科学, 2018, 45(6A): 162-166. https://doi.org/

HOU Rui-chao,ZHOU Dong-ming,NIE Ren-can,LIU Dong,GUO Xiao-peng. Infrared and Visible Images Fusion Using Visual Saliency and Dual-PCNN[J]. Computer Science, 2018, 45(6A): 162-166. https://doi.org/

参考文献

[1]JIN X,JIANG Q,YAO S,et al.A survey of infrared and visual image fusion methods[J].Infrared Physics & Technology,2017,85:478-501.
[2]LIU S,CHEN J.A fast multi-focus image fusion algorithm by DWT and focused region decision map[C]∥Signal and Information Processing Association Summit and Conference.IEEE,2017:1-7.
[3]DO M N,VETTERLI M.The contourlet transform:an efficient directional multiresolution image representation [J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2005,14(12):2091.
[4]EASLEY G,LABATE D,LIM W Q.Sparse directional image representations using the discrete shearlet transform[J].Applied & Computational Harmonic Analysis,2008,25(1):25-46.
[5]DA C A,ZHOU J,DO M N.The nonsubsampled contourlet transform:theory,design,and applications [J].IEEE Transactions on Image Processing,2006,15(10):3089-3101.
[6]HOU B,ZHANG X,BU X,et al.SAR image despeckling based on nonsubsampled shearlet transform [J].IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sen-sing,2012,5(3):809-823.
[7]LIU H,ZHU T,ZHAO J.Infrared and visible image fusion based on region of interest detection and nonsubsampled con-tourlet transform [J].Journal of Shanghai Jiaotong University,2013,18(5):526-534.
[8]ZHAO J,ZHOU Q,CHEN Y,et al.Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition [J].Infrared Physics & Technology,2013,56(2):93-99.
[9]FU Z Z,WANG X,LI X F.Infrared and visible image fusion based on visual saliency and NSCT [J].Journal of University of Electronic Science and Technology of China,2017,46(2):357-362.
[10]ECKHORN R,REITBOECK H J,ARNDT M,et al.Feature linking via synchronization among distributed assemblies:Simulations of results from cat visual cortex [J].Neural Computation,1990,2(3):293-307.
[11]KONG W,ZHANG L,LEI Y.Novel fusion method for visible light and infrared images based on NSST-SF-PCNN [J].Infrared Physics & Technology,2014,65(7):103-112.
[12]BAKIC＇ D,KRISHTAL I,WILSON E N.Parseval frame wavelets with En(2)-dilations [J].Applied & Computational Harmonic Analysis,2005,19(3):386-431.
[13]HUANG Z,DING M,ZHANG X.Medical image fusion based on non-subsampled shearlet transform and spiking cortical mo-del [J].Journal of Medical Imaging & Health Informatics,2017.
[14]ACHANTA R,HEMAMI S,ESTRADA F,et al.Frequency- tuned salient region detection[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2009(CVPR 2009).IEEE,2010:1597-1604.
[15]HE K,SUN J,TANG X.Guided image filtering [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(6):1397.
[16]ITTI L,KOCH C,NIEBUR E.A Model of saliency-based visual attention for rapid scene analysis[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,1998,20(11):1254-1259.
[17]ZHANG L,TONG M H,MARKS T K,et al.SUN:A bayesian framework for saliency using natural statistics [J].Journal of Vision,2008,8(7):1-20.
[18]HOU X,ZHANG L.Saliency Detection:A spectral residual approach[C]∥IEEE Conference on Computer Vision and Pattern Recognition,2007(CVPR’07).IEEE,2007:1-8.
[19]JIA Y,RONG C,ZHU Y,et al.Multi-focus Image Fusion Sch- eme Using Adaptive Dual-Channel Pulse Coupled Neural Network[C]∥International Conference on Intelligent Human-Machine Systems and Cybernetics.IEEE,2016:147-151.
[20]HUANG F,ZHOU D,NIE R,et al.A multi-focus color image fusion algorithm based on an adaptive SF-PCNN in NSCT domain[C]∥International Conference on Digital Image Proces-sing.2017:1042021.
[21]WANG W,CHANG F.A Multi-focus Image Fusion Method Based on Laplacian Pyramid[J].Journal of Computers,2011,6(12):2559-2566.
[22]WANG X,YAO L,SONG R,et al.A New Infrared and Visible Image Fusion Algorithm in NSCT Domain[C]∥International Conference on Intelligent Computing.Springer,Cham,2017:420-431.
[23]YANG B,LI S.Pixel-level image fusion with simultaneous orthogonal matching pursuit [J].Information Fusion,2012,13(1):10-19

相关文章 15

[1]	赵明华, 周童童, 都双丽, 石争浩. 基于虚拟曝光方法的单幅逆光图像增强 Single Backlit Image Enhancement Based on Virtual Exposure Method 计算机科学, 2022, 49(6A): 384-389. https://doi.org/10.11896/jsjkx.210400243
[2]	来腾飞, 周海洋, 余飞鸿. 视频流的实时景深延拓算法 Real-time Extend Depth of Field Algorithm for Video Processing 计算机科学, 2022, 49(6A): 314-318. https://doi.org/10.11896/jsjkx.201100187
[3]	高元浩, 罗晓清, 张战成. 基于特征分离的红外与可见光图像融合算法 Infrared and Visible Image Fusion Based on Feature Separation 计算机科学, 2022, 49(5): 58-63. https://doi.org/10.11896/jsjkx.210200148
[4]	颜敏, 罗晓清, 张战成. 基于光传输模型学习的红外和可见光图像融合网络设计 Infrared and Visible Image Fusion Network Based on Optical Transmission Model Learning 计算机科学, 2022, 49(4): 215-220. https://doi.org/10.11896/jsjkx.210200174
[5]	官铮, 邓扬琳, 聂仁灿. 光谱重建约束非负矩阵分解的高光谱与全色图像融合 Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion 计算机科学, 2021, 48(9): 153-159. https://doi.org/10.11896/jsjkx.200900054
[6]	黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合 Multi-focus Image Fusion Method Based on PCANet in NSST Domain 计算机科学, 2021, 48(9): 181-186. https://doi.org/10.11896/jsjkx.200800064
[7]	田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法 Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator 计算机科学, 2021, 48(8): 185-190. https://doi.org/10.11896/jsjkx.200600132
[8]	王丽芳, 王蕊芳, 蔺素珍, 秦品乐, 高媛, 张晋. 基于双残差超密集网络的多模态医学图像融合 Multimodal Medical Image Fusion Based on Dual Residual Hyper Densely Networks 计算机科学, 2021, 48(2): 160-166. https://doi.org/10.11896/jsjkx.200400095
[9]	朱珍, 黄锐, 臧铁钢, 卢世军. 基于加权近红外图像融合的单幅图像除雾方法 Single Image Defogging Method Based on Weighted Near-InFrared Image Fusion 计算机科学, 2020, 47(8): 241-244. https://doi.org/10.11896/jsjkx.200300068
[10]	袁野, 和晓歌, 朱定坤, 王富利, 谢浩然, 汪俊, 魏明强, 郭延文. 视觉图像显著性检测综述 Survey of Visual Image Saliency Detection 计算机科学, 2020, 47(7): 84-91. https://doi.org/10.11896/jsjkx.190900006
[11]	刘俊琦,李智,张学阳. 可见光遥感图像海面目标检测技术综述 Review of Maritime Target Detection in Visible Bands of Optical Remote Sensing Images 计算机科学, 2020, 47(3): 116-123. https://doi.org/10.11896/jsjkx.190300102
[12]	朱莹,夏亦犁,裴文江. 基于改进的BEMD的红外与可见光图像融合方法 Fusion of Infrared and Color Visible Images Based on Improved BEMD 计算机科学, 2020, 47(3): 124-129. https://doi.org/10.11896/jsjkx.190100038
[13]	李昌兴, 雷柳, 张晓璐. 基于形态学图像增强和PCNN的脑部CT与MRI图像融合 Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN 计算机科学, 2020, 47(10): 194-199. https://doi.org/10.11896/jsjkx.190700185
[14]	李笑宇,高清维,卢一相,孙冬. 一种根据图像能量调整的图像融合方法 Image Fusion Method Based on Image Energy Adjustment 计算机科学, 2020, 47(1): 153-158. https://doi.org/10.11896/jsjkx.181202437
[15]	张冰, 谢从华, 刘哲. 基于显著稀疏表示和邻域信息的多聚焦图像融合 Multi-focus Image Fusion Based on Latent Sparse Representation and Neighborhood Information 计算机科学, 2019, 46(9): 254-258. https://doi.org/10.11896/j.issn.1002-137X.2019.09.038

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed