计算机科学 ›› 2022, Vol. 49 ›› Issue (5): 58-63.doi: 10.11896/jsjkx.210200148
高元浩1,2, 罗晓清1,2, 张战成3
GAO Yuan-hao1,2, LUO Xiao-qing1,2, ZHANG Zhan-cheng3
摘要： 在同一场景下被捕获的一对红外与可见光图像虽然具有不同的模态,但是具有共享的公有信息和互补的私有信息,学习并融合上述信息可以得到一幅完整的融合图像。受益于残差网络的启发,在训练学习阶段,通过网络分支间特征层面的互换和相加,强制每一个分支映射到一幅具有全局特征的标签图上,来鼓励各个分支学习对应模态图像的私有特征。直接学习得到图像的私有特征可以避免设计复杂的融合规则并保证特征细节信息的完整。在融合预测阶段,采用最大值融合策略融合私有特征,并在解码层与学习得到的公有特征相叠加,最后解码出集成了红外和可见光图像信息的融合图像。使用在NYU-D2上合成的多聚焦图像数据集训练该模型,在TNO真实的红外和可见光数据集上进行测试,实验结果表明,与当前主流的红外与可见光融合算法相比,所提算法在主观效果和客观评价指标上都取得了较好的成绩。
中图分类号:
| [1]YU X C,GAO G Y,XU J D,et al.Remote sensing image fusion based on sparse representation[C]//2014 IEEE Geoscience and Remote Sensing Symposium.2014:2858-2861. [2]ZHAO W D,LU H C.Medical Image Fusion and Denoising withAlternating Sequential Filter and Adaptive Fractional Order Total Variation[J].IEEE Transactions on Instrumentation and Measurement,2017,66(9):2283-2294. [3]LI Y S,TAO C,TAN Y H,et al.Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification[J].IEEE Geoscience and Remote Sensing Letters,2016,13(2):157-161. [4]GUAN Z,DENG Y L,NIE R C.Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion[J].Computer Science,2021,48(9):153-159. [5]JIN X,JIANG Q,YAO S W,et al.A survey of infrared and vi-sual image fusion methods[J].Information Fusion,2017,85:478-501. [6]BAI X Z,ZHANG Y,ZHOU F G,et al.Quadtree-based multi-focus image fusion using a weighted focus-measure[J].Information Fusion,2015,22:105-118. [7]WANG W C,CHANG F L.A multi-focus image fusion method based on laplacian pyramid[J].Journal of Computers,2011,6(12),2559-2566. [8]RONNEBERGER O,FISCHER P,BROX T,et al.U-net:Con-volutional networks for biomedical image segmentation[J].Lecture Notes in Computer Science,2015,9351:234-241. [9]PAUL V,JONATHON L,PHILIP H S,et al.Siam R-CNN:Visual Tracking by Re-Detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).2020. [10]LIU Y,CHEN X,PENG H,et al.Multi-focus image fusion with a deep convolutional neural network[J].Information Fusion,2017,36:191-207. [11]LI H,WU X J.Densefuse:A fusion approach to infrared and vi-sible images[J].IEEE Transactions on Image Processing,2019,28(5):2614-2623. [12]ZHANG Y,LIU Y,SUN P,et al.IFCNN:A general image fusion framework based on convolutional neural network[J].Information Fusion.2020,54:99-118. [13]MA J Y,XU H,JIANG J J,et al.DDcGAN:A Dual-discriminator Conditional Generative Adversarial Network for Multi-resolution Image Fusion[J].IEEE Transactions on Image Proces-sing,2020,29:4980-4995. [14]FU Y,WU X J.Image fusion based on generative adversarial network consistent with perception[J].Information Fusion,2021,72:110-125. [15]HE K M,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.2016:770-778. [16]LI H,WU X J.Infrared and visible image fusion with ResNet and zero-phase component analysis[OL].https://arxiv.org/pdf/1806.07119.pdf. [17]LIU Y,LIU S P,WANG Z F,et al.A General Framework for Image Fusion Based on Multi-scale Transform and Sparse Representation[J].Information Fusion.2015,24:147-164. [18]MA J Y,CHEN C,LI C,et al.Infrared and visible image fusion via gradient transfer and total variation minimization[J].Information Fusion,2016,31(C):100-109. [19]PRABHAKAR K R,SRIKAR V S,BABU R V,et al.Deepfuse:A deep unsupervised approach for exposure fusion with extreme exposure image pairs[C]//IEEE International Conference on Computer Vision.2017:4724-4732. [20]MA J Y,YU W,LIANG P W,et al.Fusiongan:A generativeadversarial network for infrared and visible image fusion[J].Information Fusion,2019,48:11-26. [21]LIU Z,BLASCH E,XUE Z,et al.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision:A comparative study[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,34(1):94-109. [22]WANG Z,BOVIK A C.A universal image quality index[J].IEEE Signal Processing Letters,2002,9(3):81-84. |
| [1] | 张源, 康乐, 宫朝辉, 张志鸿. 基于Bi-LSTM的期货市场关联交易行为检测方法 Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM 计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304 |
| [2] | 曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨. 基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨 Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism 计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224 |
| [3] | 程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157 |
| [4] | 刘伟业, 鲁慧民, 李玉鹏, 马宁. 指静脉识别技术研究综述 Survey on Finger Vein Recognition Research 计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056 |
| [5] | 来腾飞, 周海洋, 余飞鸿. 视频流的实时景深延拓算法 Real-time Extend Depth of Field Algorithm for Video Processing 计算机科学, 2022, 49(6A): 314-318. https://doi.org/10.11896/jsjkx.201100187 |
| [6] | 赵明华, 周童童, 都双丽, 石争浩. 基于虚拟曝光方法的单幅逆光图像增强 Single Backlit Image Enhancement Based on Virtual Exposure Method 计算机科学, 2022, 49(6A): 384-389. https://doi.org/10.11896/jsjkx.210400243 |
| [7] | 颜敏, 罗晓清, 张战成. 基于光传输模型学习的红外和可见光图像融合网络设计 Infrared and Visible Image Fusion Network Based on Optical Transmission Model Learning 计算机科学, 2022, 49(4): 215-220. https://doi.org/10.11896/jsjkx.210200174 |
| [8] | 左杰格, 柳晓鸣, 蔡兵. 基于图像分块与特征融合的户外图像天气识别 Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion 计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263 |
| [9] | 任首朋, 李劲, 王静茹, 岳昆. 基于集成回归决策树的lncRNA-疾病关联预测方法 Ensemble Regression Decision Trees-based lncRNA-disease Association Prediction 计算机科学, 2022, 49(2): 265-271. https://doi.org/10.11896/jsjkx.201100132 |
| [10] | 官铮, 邓扬琳, 聂仁灿. 光谱重建约束非负矩阵分解的高光谱与全色图像融合 Non-negative Matrix Factorization Based on Spectral Reconstruction Constraint for Hyperspectral and Panchromatic Image Fusion 计算机科学, 2021, 48(9): 153-159. https://doi.org/10.11896/jsjkx.200900054 |
| [11] | 黄晓生, 徐静. 基于PCANet的非下采样剪切波域多聚焦图像融合 Multi-focus Image Fusion Method Based on PCANet in NSST Domain 计算机科学, 2021, 48(9): 181-186. https://doi.org/10.11896/jsjkx.200800064 |
| [12] | 张师鹏, 李永忠. 基于降噪自编码器和三支决策的入侵检测方法 Intrusion Detection Method Based on Denoising Autoencoder and Three-way Decisions 计算机科学, 2021, 48(9): 345-351. https://doi.org/10.11896/jsjkx.200500059 |
| [13] | 田嵩旺, 蔺素珍, 杨博. 基于多判别器的多波段图像自监督融合方法 Multi-band Image Self-supervised Fusion Method Based on Multi-discriminator 计算机科学, 2021, 48(8): 185-190. https://doi.org/10.11896/jsjkx.200600132 |
| [14] | 冯霞, 胡志毅, 刘才华. 跨模态检索研究进展综述 Survey of Research Progress on Cross-modal Retrieval 计算机科学, 2021, 48(8): 13-23. https://doi.org/10.11896/jsjkx.200800165 |
| [15] | 张丽倩, 李孟航, 高珊珊, 张彩明. 面向计算机辅助舌诊关键问题的解决方案综述 Summary of Computer-assisted Tongue Diagnosis Solutions for Key Problems 计算机科学, 2021, 48(7): 256-269. https://doi.org/10.11896/jsjkx.200800223 |
|
||
首页