基于FPDEs与CBF的红外与可见光图像融合

doi:10.11896／j.issn.1002-137X.2019.01.046

摘要/Abstract

摘要： 针对传统红外与可见光图像融合结果中的对比度不足、块状效应、伪影以及边缘区域信息失真等问题,文中提出一种基于四阶偏微分方程(FPDEs)和交叉双边滤波器(CBF)的红外与可见光图像融合方法。首先,分别使用FPDEs和CBF方法从源图像中获取近似层和细节层;其次,针对多尺度分解获得的近似层含有残余低频信息导致融合图像的整体视觉反差较大的问题,采用基于视觉显著性映射(VSM)的方法对近似层进行融合;然后,对细节层使用改进的Karhunen-Loeve变换获得权重,而后进行细节层融合;最后,通过线性组合方式将近似层与细节层融合,从而产生融合图像。实验结果表明,经基于FPDEs与CBF的方法融合后,相较于基于主成分分析和基于交叉双边滤波器的方法,基于FPDEs与CBF的方法所得融合图像的标准差平均提高了43.73%左右;相较于基于引导滤波器和基于视觉显著性最小二乘优化的方法,融合图像的平均梯度提高了约9.46%,空间频率平均提高了19.79%左右。

关键词: Karhunen-Loeve变换, 交叉双边滤波器, 视觉显著性映射, 四阶偏微分方程, 图像融合

Abstract: Considering the problems of low contrast,blocky effects,artifacts and distortion of the edge region in traditional fused images,this paper proposed an infrared image and visible image fusion method based on fourth order partial differential equations(FPDEs) and cross bilateral filter(CBF).Firstly,the FPDEs and CBF are respectively used to obtain the approximation layers and detail layers from the source image.Secondly,the approximate layers obtained by multi-scale decompositions may contain amount of residual low-frequence information which will result in large contrast of the overall visual of the fused image,so a fusion method based on visual saliency map(VSM) is used to fuse the approximate layers.Thirdly,an improved Karhunen-Loeve transform is applied into the detail layer to obtain the optimal weights for fusion.Finally,a fused image is generated from the linear combination of final approximate layers and detail layers.Experimental results show that the standard deviation of the fused image obtained by the proposed method increases about 43.3% than PCA based method and cross bilateral filter based method,and the average gradient and spatial frequency increase about 9.46% and 19.79% respectively on average compared with GFF and VSM_WLS algorithms.

Key words: Cross bilateral filter, Fourth order partial differential equations, Image fusion, Karhunen-Loeve transform, Visual saliency map

中图分类号:

TP391

李昌兴, 武洁. 基于FPDEs与CBF的红外与可见光图像融合[J]. 计算机科学, 2019, 46(1): 297-302. https://doi.org/10.11896／j.issn.1002-137X.2019.01.046

LI Chang-xing, WU Jie. Infrared Image and Visible Image Fusion Based on FPDEs and CBF[J]. Computer Science, 2019, 46(1): 297-302. https://doi.org/10.11896／j.issn.1002-137X.2019.01.046

参考文献

[1]DU L.Infrared and visible image fusion based on random projection and sparse representation[J].International Journal of Remote Sensing,2014,35(5):1640-1652. [2]GHASSEMIAN H.A review of remote sensing image fusion methods[J].Information Fusion,2016,32(PA):75-89. [3]ZHOU Y R,GENG A H,ZHANG Q,et al.Fusion of infrared and visible images based on compressive sensing[J].Optics and Precision Engineering,2015,23(3):855-863.(in Chinese) 周渝人,耿爱辉,张强,等.基于压缩感知的红外与可见光图像融合[J].光学精密工程,2015,23(3):855-863. [4]LI C X,WANG P P,SHEN C.The fusion of medical images based on two transform domains[J].Journal of Xi’an University of Posts and Telecommunications,2013,18(4):37-40.(in Chinese) 李昌兴,王平平,申超.两种变换域相结合的医学图像融合[J].西安邮电大学学报,2013,18(4):37-40. [5]KUMAR B K S.Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform[J].Signal Image & Video Processing,2013,7(6):1125-1143. [6]KUMAR B K S.Image fusion based on pixel significance using cross bilateral filter[J].Signal Image & Video Processing,2015,9(5):1193-1204. [7]LI S,KANG X,HU J.Image fusion with guided filtering[J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2013,22(7):2864. [8]BAVIRISETTI,PRASAD D,DHULI R-V.Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform[J].IEEE Sensors Journal,2016,16(1):203-209. [9]YOU Y L,KAVEH M.Fourth-order partial differential equations for noise removal[J].IEEE Transactions on Image Processing,2000,9(10):1723-1730. [10]ZHAI Y,SHAH M.Visual attention detection in video sequences using spatiotemporal cues[C]//ACM International Conference on Multimedia.ACM,2006:815-824. [11]GONZALEZ R C.Digital Image Processing[M].Chandigarh,India:Pearson Education India,2009. [12]THEOHARATOS C,TSAGARIS V,FRAGOULIS N,et al.Hyperspectral image fusion using 2-D principal component ana-lysis[C]//International Conference on Space Technology.IEEE,Athens,2011:1-4. [13]VIJAYARAJAN R,MUTTAN S.Discrete wavelet transform based principal component averaging fusion for medical images[J].AEUE-International Journal of Electronics and Communications,2015,69(6):896-902. [14]MA J,ZHOU Z,WANG B,et al.Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J].Infrared Physics & Technology,2017,82:8-17. [15]BAVIRISETTI D P,XIAO G,LIU G.Multi-sensor image fusion based on fourth order partial differential equations[C]//Proceedings of IEEE 2017 20th International Conference on Information Fusion (Fusion).Xi’an,2017:1-9. [16]ÅSTRÖM F,BARAVDISH G,FELSBERG M.On Tensor-Based PDEs and Their Corresponding Variational Formulations with Application to Color Image Denoising[M]//Computer Vision-ECCV 2012.Springer Berlin Heidelberg,2012:215-228. [17]PERONA P,MALIK J.Scale-space and edge detection using anisotropic diffusion[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,12(7):629-639. [18]TOMASI C,MANDUCHI R.Bilateral filtering for gray and color images[C]//Proceedings of the International Conference on Computer Vision.1998:839-846. [19]ZHANG M,GUNTURK B K.Multiresolution bilateral filtering for image denoising[J].IEEE Transactions on Image Proces-sing,2008,17(12):2324-2333. [20]PETSCHNIGG G,AGRAWALA M,HOPPE H,et al.Digital photography with flash and no-flash image pairs[J].ACM Transactions on Graphics,2004,23(3):664-672. [21]CHENG M M,MITRA N J,HUANG X,et al.Global Contrast Based Salient Region Detection[J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,37(3):569. [22]BAI X,CHEN X,ZHOU F,et al.Multiscale top-hat selection transform based infrared and visual image fusion with emphasis on extracting regions of interest[J].Infrared Physics & Technology,2013,60(9):81-93. [23]HAGHIGHAT M B A,AGHAGOLZADEH A,SEYEDARABI H.A non-reference image fusion metric based on mutual information of image features[J].Computers & Electrical Enginee-ring,2011,37(5):744-756.

相关文章 15

[1]	来腾飞, 周海洋, 余飞鸿. 视频流的实时景深延拓算法 Real-time Extend Depth of Field Algorithm for Video Processing 计算机科学, 2022, 49(6A): 314-318. https://doi.org/10.11896/jsjkx.201100187
[2]	赵明华, 周童童, 都双丽, 石争浩. 基于虚拟曝光方法的单幅逆光图像增强 Single Backlit Image Enhancement Based on Virtual Exposure Method 计算机科学, 2022, 49(6A): 384-389. https://doi.org/10.11896/jsjkx.210400243
[3]	高元浩, 罗晓清, 张战成. 基于特征分离的红外与可见光图像融合算法 Infrared and Visible Image Fusion Based on Feature Separation 计算机科学, 2022, 49(5): 58-63. https://doi.org/10.11896/jsjkx.210200148
[4]	颜敏, 罗晓清, 张战成. 基于光传输模型学习的红外和可见光图像融合网络设计 Infrared and Visible Image Fusion Network Based on Optical Transmission Model Learning 计算机科学, 2022, 49(4): 215-220. https://doi.org/10.11896/jsjkx.210200174
[5]	官铮, 邓扬琳, 聂仁灿. 光谱重建约束非负矩阵分解的高光谱与全色图像融合 Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion 计算机科学, 2021, 48(9): 153-159. https://doi.org/10.11896/jsjkx.200900054
[6]	黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合 Multi-focus Image Fusion Method Based on PCANet in NSST Domain 计算机科学, 2021, 48(9): 181-186. https://doi.org/10.11896/jsjkx.200800064
[7]	田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法 Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator 计算机科学, 2021, 48(8): 185-190. https://doi.org/10.11896/jsjkx.200600132
[8]	王丽芳, 王蕊芳, 蔺素珍, 秦品乐, 高媛, 张晋. 基于双残差超密集网络的多模态医学图像融合 Multimodal Medical Image Fusion Based on Dual Residual Hyper Densely Networks 计算机科学, 2021, 48(2): 160-166. https://doi.org/10.11896/jsjkx.200400095
[9]	朱珍, 黄锐, 臧铁钢, 卢世军. 基于加权近红外图像融合的单幅图像除雾方法 Single Image Defogging Method Based on Weighted Near-InFrared Image Fusion 计算机科学, 2020, 47(8): 241-244. https://doi.org/10.11896/jsjkx.200300068
[10]	朱莹,夏亦犁,裴文江. 基于改进的BEMD的红外与可见光图像融合方法 Fusion of Infrared and Color Visible Images Based on Improved BEMD 计算机科学, 2020, 47(3): 124-129. https://doi.org/10.11896/jsjkx.190100038
[11]	李昌兴, 雷柳, 张晓璐. 基于形态学图像增强和PCNN的脑部CT与MRI图像融合 Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN 计算机科学, 2020, 47(10): 194-199. https://doi.org/10.11896/jsjkx.190700185
[12]	李笑宇,高清维,卢一相,孙冬. 一种根据图像能量调整的图像融合方法 Image Fusion Method Based on Image Energy Adjustment 计算机科学, 2020, 47(1): 153-158. https://doi.org/10.11896/jsjkx.181202437
[13]	张冰, 谢从华, 刘哲. 基于显著稀疏表示和邻域信息的多聚焦图像融合 Multi-focus Image Fusion Based on Latent Sparse Representation and Neighborhood Information 计算机科学, 2019, 46(9): 254-258. https://doi.org/10.11896/j.issn.1002-137X.2019.09.038
[14]	王丽芳, 史超宇, 蔺素珍, 秦品乐, 高媛. 基于联合图像块聚类自适应字典学习的多模态医学图像融合 Multi-modal Medical Image Fusion Based on Joint Patch Clustering of Adaptive Dictionary Learning 计算机科学, 2019, 46(7): 238-245. https://doi.org/10.11896/j.issn.1002-137X.2019.07.036
[15]	刘斌,谌文江,辛迦楠. 基于四通道不可分加性小波的多聚焦图像融合 Multifocus Image Fusion Based on Four Channel Non-separable Additive Wavelet 计算机科学, 2019, 46(7): 268-273. https://doi.org/10.11896/j.issn.1002-137X.2019.07.041

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed