一种根据图像能量调整的图像融合方法

doi:10.11896/jsjkx.181202437

摘要/Abstract

摘要： 针对传统图像融合算法无法对能量差异较大的图像取得良好融合效果的问题,文中根据图像的能量划分,利用多尺度变换和稀疏表示相结合的方式分解两幅图像的高低频信号,在低频部分自动调整不同能量图像块的稀疏融合规则,并在高频部分加入一致性检验,从而进一步约束对应局部空间能量MSD系数的复合过程,最后通过小波逆变换重构得到融合图像。使用红外图像、医学图像和多聚焦图像分别进行融合性能的验证,并分析稀疏分解层数和窗口步长等条件对融合效果的影响,最终取得该框架下的最优分解方式,获得了具备优秀的主观效果和客观指标的融合图像。实验结果表明,该算法在对任意两种类型传感器获得的图像进行融合时均能获得更加优秀的融合效果,且不仅局限于某两种图像的融合,其在SF,SSIM和EFQI等客观指标上优于传统融合算法和一般多尺度结合稀疏表示的算法。

关键词: 多尺度变换, 能量划分, 图像融合, 稀疏表示, 一致性检验

Abstract: Aiming at the problem that traditional image fusion algorithm can’t achieve good fusion effect on the image with large energy difference,this paper decomposed the high and low frequency signals of the two images by the combination of multi-scale transformation and sparse representation according to the energy division of the image.The sparse fusion rules of different energy image blocks are adjusted,and the consistency test is added in the high frequency part to further constrain the composite process of the MSD coefficients corresponding to the local spatial energy.Finally,the fused image is reconstructed by wavelet inverse transform.Infrared images,medical images and multi-focus images are used to verify its performance,the effects of sparse decomposition layer number and window step size on the fusion effect are analyzed,the optimal decomposition method under the framework is obtained,and then the fusion image with excellent subjective results and objective inclications are obtained.The experimental results show that the proposed algorithm can achieve better fusion effect when the image is obtained by any two types of sensors,and is not limited to the fusion of two images.It is superior to the traditional indicators such as SF,SSIM and EFQI of fusion algorithm and general multi-scale algorithm combined with sparse representation.

Key words: Consistency test, Energy division, Image fusion, Multi-scale transformation, Sparse representation

中图分类号:

TP391

李笑宇,高清维,卢一相,孙冬. 一种根据图像能量调整的图像融合方法[J]. 计算机科学, 2020, 47(1): 153-158. https://doi.org/10.11896/jsjkx.181202437

LI Xiao-yu,GAO Qing-wei,LU Yi-xiang,SUN Dong. Image Fusion Method Based on Image Energy Adjustment[J]. Computer Science, 2020, 47(1): 153-158. https://doi.org/10.11896/jsjkx.181202437

参考文献

[1]PSJR C,SIDES S C,ANDERSON J A.Comparison of three different methods to merge multiresolution and multispectral data:Landsat TM and SPOT panchromatic[J].Photogrammetric Engineering & Remote Sensing,1991,57(3):265-303.
[2]RUI G,VOROBYOV S,HONG Z.Image FusionWith Cosparse Analysis Operator[J].IEEE Signal Processing Letters,2017,PP(99):1-1.
[3]RATTÁ G A,VEGA J,MURARI A,et al.Image fusion:Advances in the state of the art[J].Information Fusion,2007,8(2):114-118.
[4]WEN X.Image Fusion Based on Improved IHS Transform with Weighted Average[C]∥ International Conference on Computational & Information Sciences.Chengdu:IEEE,2011.
[5]HERVÉ A,WILLIAMS L J.Principal component analysis[J].Wiley Interdisciplinary Reviews Computational Statistics,2010,2(4):433-459.
[6]CUI Z,ZHANG G,WU J.Medical Image Fusion Based on Wavelet Transform and Independent Component Analysis[C]∥International Joint Conference on Artificial Intelligence.Hainan:IEEE,2009.
[7]YANG L,GUO B L,NI W.Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform[J].Neurocomputing,2008,72(1):203-211.
[8]BURT P J,ADELSON E H.The Laplacian Pyramid as a Com- pact Image Code[J].Readings in Computer Vision,1987,31(4):671-679.
[9]PETROVIC V S,XYDEAS C S.Gradient-Based Multiresolution Image Fusion[J].IEEE Transactions on Image Processing,2004,13(2):228-237.
[10]LI H,MANJUNATH B S,MITRA S K.Multi-Sensor Image Fusion using the Wavelet Transform[C]∥ Image Processing,Icip-94,IEEE International Conference.New York:IEEE,2002.
[11]LI S,KWOK J T,WANG Y.Using the discrete wavelet frame transform to merge Landsat TM and SPOT panchromatic images[J].Information Fusion,2002,3(1):17-23.
[12]ZHANG Q,GUO B L.Multifocus image fusion using the nonsubsampled contourlet transform[J].Signal Processing,2009,89(7):1334-1346.
[13]XIAO D J,LI Q S,ZHOU C L.Research on adaptive image fusion method based on sparse representation[J].Computer Applications And Software,2014(3):203-206.
[14]YANG J,WRIGHT J,HUANG T S,et al.Image Super-Resolution Via Sparse Representation[J].IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[15]REXILINE D N D,ANUSMINA D J.Fusion and restoration of multifocus image using sparse representation[C]∥ International Conference on Advances in Engineering.Zouk-Mosbeh:IEEE,2012.
[16]REN Z,WANG S H,YUE L H.Source Optimization Selection Method for Multiple Heterogeneous Image Fusion[J].Computer Engineering,2009,35(23):229-231.
[17]LI S,YANG B,HU J.Performance comparison of different multi-resolution transforms for image fusion[J].Information Fusion,2011,12(2):74-84.
[18]PATI Y C,REZAIIFAR R,KRISHNAPRASAD P S.Orthogonal matching pursuit:recursive function approximation with applications to wavelet decomposition[C]∥ Conference on Signals,Systems & Computers.California:IEEE,2002.
[19]ZONG J,QIU T.Brain Image Fusion Based on Online Dictionary Learning and Pulse Coupled Neural Network[J].Chinese Journal of Biomedical Engineering,2015,5(3):223-227.
[20]GAO G,XU L,FENG D.Multi-focus image fusion based on non-subsampled shearlet transform[J].IET Image Processing,2013,7(6):633-639.
[21]SUN W.Application of shear wave transform in image fusion [D].Beijing:Beijing Jiaotong University,2015.
[22]XU Z Z,QU S R.A New Comprehensive Evaluation of Visible and Infrared Image Fusion[J].Infrared Technology,2011,33(10):568-573.
[23]HAN J J,LOFFELD O,HARTMANN K,et al.Multi image fusion based on compressive sensing[C]∥Proc of IEEE International Conference on Audio Language and Image Processing.2010:1463-1469.
[24]LIU Y,LIU S,WANG Z.A general framework for image fusion based on multi-scale transform and sparse representation[J].Information Fusion,2015,24(1):147-164.
[25]CAO Y Q,CAO T,HUANG X S.Image fusion method combined CS and regional characteristics based on NSST[J].Computer Engineering and Applications,2018,54(20):190-196.
[26]CHEN J Q,HUANG D F.A Medical Image Fusion Improved Algorithm Based on NSCT and Adaptive PCNN[J].Journal of Changchun University of Science and Technology(Natural Science Edition),2015,38(3):152-155,159.

相关文章 15

[1]	来腾飞, 周海洋, 余飞鸿. 视频流的实时景深延拓算法 Real-time Extend Depth of Field Algorithm for Video Processing 计算机科学, 2022, 49(6A): 314-318. https://doi.org/10.11896/jsjkx.201100187
[2]	赵明华, 周童童, 都双丽, 石争浩. 基于虚拟曝光方法的单幅逆光图像增强 Single Backlit Image Enhancement Based on Virtual Exposure Method 计算机科学, 2022, 49(6A): 384-389. https://doi.org/10.11896/jsjkx.210400243
[3]	高元浩, 罗晓清, 张战成. 基于特征分离的红外与可见光图像融合算法 Infrared and Visible Image Fusion Based on Feature Separation 计算机科学, 2022, 49(5): 58-63. https://doi.org/10.11896/jsjkx.210200148
[4]	颜敏, 罗晓清, 张战成. 基于光传输模型学习的红外和可见光图像融合网络设计 Infrared and Visible Image Fusion Network Based on Optical Transmission Model Learning 计算机科学, 2022, 49(4): 215-220. https://doi.org/10.11896/jsjkx.210200174
[5]	官铮, 邓扬琳, 聂仁灿. 光谱重建约束非负矩阵分解的高光谱与全色图像融合 Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion 计算机科学, 2021, 48(9): 153-159. https://doi.org/10.11896/jsjkx.200900054
[6]	黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合 Multi-focus Image Fusion Method Based on PCANet in NSST Domain 计算机科学, 2021, 48(9): 181-186. https://doi.org/10.11896/jsjkx.200800064
[7]	田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法 Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator 计算机科学, 2021, 48(8): 185-190. https://doi.org/10.11896/jsjkx.200600132
[8]	李培冠, 於志勇, 黄昉菀. 基于稀疏表示的电力负荷数据补全 Power Load Data Completion Based on Sparse Representation 计算机科学, 2021, 48(2): 128-133. https://doi.org/10.11896/jsjkx.191200152
[9]	王丽芳, 王蕊芳, 蔺素珍, 秦品乐, 高媛, 张晋. 基于双残差超密集网络的多模态医学图像融合 Multimodal Medical Image Fusion Based on Dual Residual Hyper Densely Networks 计算机科学, 2021, 48(2): 160-166. https://doi.org/10.11896/jsjkx.200400095
[10]	田旭, 常侃, 黄升, 覃团发. 基于残差字典及协作表达的单图像超分辨率算法 Single Image Super-resolution Algorithm Using Residual Dictionary and Collaborative Representation 计算机科学, 2020, 47(9): 135-141. https://doi.org/10.11896/jsjkx.190600146
[11]	朱珍, 黄锐, 臧铁钢, 卢世军. 基于加权近红外图像融合的单幅图像除雾方法 Single Image Defogging Method Based on Weighted Near-InFrared Image Fusion 计算机科学, 2020, 47(8): 241-244. https://doi.org/10.11896/jsjkx.200300068
[12]	程中建, 周双娥, 李康. 基于多尺度自适应权重的稀疏表示目标跟踪算法 Sparse Representation Target Tracking Algorithm Based on Multi-scale Adaptive Weight 计算机科学, 2020, 47(6A): 181-186. https://doi.org/10.11896/JsJkx.190500093
[13]	吴庆洪, 高晓东. 稀疏表示和支持向量机相融合的非理想环境人脸识别 Face Recognition in Non-ideal Environment Based on Sparse Representation and Support Vector Machine 计算机科学, 2020, 47(6): 121-125. https://doi.org/10.11896/jsjkx.190500058
[14]	朱莹,夏亦犁,裴文江. 基于改进的BEMD的红外与可见光图像融合方法 Fusion of Infrared and Color Visible Images Based on Improved BEMD 计算机科学, 2020, 47(3): 124-129. https://doi.org/10.11896/jsjkx.190100038
[15]	李昌兴, 雷柳, 张晓璐. 基于形态学图像增强和PCNN的脑部CT与MRI图像融合 Brain CT and MRI Image Fusion Based on Morphological Image Enhancement and PCNN 计算机科学, 2020, 47(10): 194-199. https://doi.org/10.11896/jsjkx.190700185

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed