计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (10): 68-76.doi: 10.11896/jsjkx.260500078

• 大模型与多模态智能感知 • 上一篇    下一篇

HLDNet:基于高低频双流解耦的红外可见光图像融合网络

李烨, 王琦, 夏靖远, 黄星桦, 姜卫东   

  1. 国防科技大学电子科学学院 长沙 410073
  • 收稿日期:2026-05-14 修回日期:2026-08-03 发布日期:2026-10-13
  • 通讯作者: 夏靖远(j.xia10＠nudt.edu.cn)
  • 作者简介:(liye19@nudt.edu.cn)
  • 基金资助:
    国家自然科学基金(62531026);国家自然科学基金青年学生基础研究项目(625B2180)

HLDNet:High-Low Frequency Dual-stream Decoupling Network for Infrared and Visible ImageFusion

LI Ye, WANG Qi, XIA Jingyuan, HUANG Xinghua, JIANG Weidong   

  1. College of Electronic Science,National University of Defense Technology,Changsha 410073, China
  • Received:2026-05-14 Revised:2026-08-03 Online:2026-10-13
  • About author:LI Ye,born in 2001,Ph.D.His main research interests include multi-modal image fusion and deep learning.
    XIA Jingyuan,born in 1991,associate professor.His main research interests include optimization,AIGC and low le-vel vision.
  • Supported by:
    National Natural Science Foundation of China(62531026) and Basic Research Program for Young Students of the National Natural Science Foundation of China(625B2180).

PDF (PC)

85

摘要： 红外和可见光图像融合的目的在于融合红外图像清晰的轮廓结构和可见光图像丰富的纹理信息来得到一张融合图像,该图像拥有两种模态的互补优势特性,可用于医疗诊断、自动驾驶等场景。现有深度融合方法虽然已逐步引入频域变换、频率注意力或频谱约束,但多数工作仍将频域信息作为辅助增强项嵌入统一主干,缺少对低频结构与高频细节的显式分离和差异化建模。由此,提出一种基于高低频双流解耦的红外与可见光图像融合网络(High-Low Frequency Dual-stream Decoupling Network,HLDNet)。首先,利用高斯低通与残差高通将红外图像和可见光图像分解为低频结构分量与高频细节分量;随后,构建低频流和高频流两个异构分支来分别承担主体结构建模和局部边缘纹理增强,在融合阶段,低频流采用跨注意力引导与门控机制突出热目标和场景结构,高频流采用差分驱动的细节掩码强化边界与纹理;最后,通过跨流一致性模块,利用低频空间先验增强主体结构和显著区域,并利用高频边界信息增强图像细节纹理。与其他先进方法的对比实验表明,所提方法在ROADSCENE数据集上的MI和Qabf分别达到4.21和0.70,在M3FD数据集上的EN和SF分别达到7.30和17.99,在MSRS数据集上的VIF和Qabf分别达到1.12和0.75。实验结果验证了所提方法在边缘增强和细节保真方面的有效性。

关键词: 多模态图像, 多频融合, 高低频解耦, 跨注意力机制, 自适应重建, 红外与可见光图像融合

Abstract: Infrared and visible image fusion aims to fuse the sharp contour structures of infrared images and the rich texture information of visible images,which can be used in scenarios such as medical diagnosis and autonomous driving.Although existing deep fusion methods have gradually incorporated frequency domain transformation or frequency attention,most of them still embed frequency domain information as an auxiliary enhancement item into a unified backbone,lacking explicit separation and diffe-rentiated modeling of low-frequency structures and high-frequency details.Therefore,this paper proposes a high-low frequency dual-stream decoupling network(HLDNet) for infrared and visible image fusion.Firstly,the network decomposes infrared and visible images into low-frequency structural components and high-frequency detail components using Gaussian low-pass and residual high-pass filters.Subsequently,it constructs two heterogeneous branches,which are responsible for modeling the structural information and enhancing local edge textures,respectively.In the fusion stage,the low-frequency stream employs cross-modal guided attention and gating mechanisms to highlight thermal targets and scene structures,while the high-frequency stream utilizes a differential-driven detail mask to enhance boundaries and textures.Finally,a cross stream consistency module is employed to enhance the structural information and salient regions using low-frequency spatial priors,and to enhance image detail textures using high-frequency boundary information.Experimental comparisons with other advanced methods show that the proposed method achieves MI and Qabf of 4.21 and 0.70 on the ROADSCENE dataset,EN and SF of 7.30 and 17.99 on the M3FD dataset,and VIF and Qabf of 1.12 and 0.75 on the MSRS dataset,respectively,validating the effectiveness of the proposed method in edge enhancement and detail fidelity.

Key words: Multi-modal images, Multi-frequency fusion, High-low frequency decoupling, Cross-attention mechanism, Adaptive reconstruction, Infrared and visible image fusion

中图分类号: 

  • TP391
[1] ZHAO Z X,BAI H W,ZHANG J S,et al.CDDFuse:Correlation-driven dual-branch feature decomposition for multi-modality image fusion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:5906-5916.
[2] LIU J Y,WU G Y,LIU Z,et al.Infrared and visible image fusion:From data compatibility to task adaption[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2025,47(4):2349-2369.
[3] MA J Y,YU W,LIANG P W,et al.FusionGAN:A generative adversarial network for infrared and visible image fusion[J].Information Fusion,2019,48:11-26.
[4] MA J Y,LIANG P W,YU W,et al.Infrared and visible image fusion via detail preserving adversarial learning[J].Information Fusion,2020,54:85-98.
[5] WANG Z S,ZHANG Z Q,QI W Q,et al.FreqGAN:Infrared and visible image fusion via unified frequency adversarial lear-ning[J].IEEE Transactions on Circuits and Systems for Video Technology,2025,35(1):728-740.
[6] XU H,MA J Y,JIANG J J,et al.U2Fusion:A unified unsupervised image fusion network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(1):502-518.
[7] ZHANG H,MA J Y.SDNet:A versatile squeeze-and-decomposition network for real-time image fusion[J].International Journal of Computer Vision,2021,129(10):2761-2785.
[8] HO J,JAIN A,ABBEEL P.Denoisingdiffusion probabilisticmodels[C]//Advances in Neural Information Processing Systems.2020:6840-6851.
[9] ZHAO Z X,BAI H W,ZHU Y Z,et al.DDFM:Denoising diffusion model for multi-modality image fusion[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:8082-8093.
[10] HUANG J J,YAN P X,LIU J Y,et al.DreamFuse:Adaptiveimage fusion with diffusion transformer[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2025:17292-17301.
[11] YI X P,TANG L F,ZHANG H,et al.Diff-IF:Multi-modality image fusion via diffusion model with fusion knowledge prior[J].Information Fusion,2024,110:102450.
[12] LI H,WU X J.DenseFuse:A fusion approach to infrared andvisible images[J].IEEE Transactions on Image Processing,2019,28(5):2614-2623.
[13] LIANG P W,JIANG J J,LIU X M,et al.Fusion from decomposition:A self-supervised decomposition approach for image fusion[C]//European Conference on Computer Vision.Cham:Springer,2022:719-735.
[14] BAI H W,ZHAO Z X,ZHANG J S,et al.ReFusion:Learning image fusion from reconstruction with learnable loss via meta-learning[J].International Journal of Computer Vision,2025,133(5):2547-2567.
[15] WANG Q H,LI Z W,ZHANG S Q,et al.WaveFusion:A novelwavelet vision transformer with saliency-guided enhancement for multimodal image fusion[J].IEEE Transactions on Circuits and Systems for Video Technology,2025,35(8):7526-7542.
[16] CHEN H R,DENG L,CHEN Z X,et al.SFCFusion:Spatial-frequency collaborative infrared and visible image fusion[J].IEEE Transactions on Instrumentation and Measurement,2024,73:1-15.
[17] GOODFELLOW I J,POUGET-ABADIE J,MIRZA M,et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems.2014.
[18] HEIL C E,WALNUT D F.Continuous and discrete wavelettransforms[J].SIAM Review,1989,31(4):628-666.
[19] VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems.2017.
[20] VILALTA R,DRISSI Y.A perspective view and survey of meta-learning[J].Artificial Intelligence Review,2002,18(2):77-95.
[21] LIU J Y,FAN X,HUANG Z B,et al.Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5802-5811.
[22] ZHAO Z X,XU S,ZHANG C X,et al.DIDFuse:Deep image de-composition for infrared and visible image fusion[C]//Procee-dings of the International Joint Conference on Artificial Intelligence.2020:970-976.
[23] HE D,LI W S,WANG G F,et al.MMIF-INet:Multimodal me-dical image fusion by invertible network[J].Information Fusion,2025,114:102666.
[24] JIANG Q,ZHOU T,YU D J,et al.GSU-Fusion:A text-guided symmetric U-Net framework for infrared and visible image fusion[J].Information Fusion,2026,127:103894.
[25] YANG C J,LUO X Q,ZHANG Z C,et al.KDFuse:A high-level vision task-driven infrared and visible image fusion method basedon cross-domain knowledge distillation[J].Information Fusion,2025,118:102944.
[26] ZHAO Z X,BAI H W,ZHANG J S,et al.Equivariant multi-modality image fusion[C]//Proceedings of the IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition.2024.
[27] TANG L F,YUAN J T,ZHANG H T,et al.PIAFusion:A progressive infrared and visible image fusion network based on illumination aware[J].Information Fusion,2022,83-84:79-92.
[28] XU H,MA J Y,LE Z L,et al.FusionDN:A unified densely connected network for image fusion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12484-12491.
[29] SUMMERS D.Harvard Whole Brain Atlas:www.med.har-vard.edu/AANLIB/home.html[J].Journal of Neurology,Neurosurgery & Psychiatry,2003,74(3):288.
[30] KINGMA D P,BA J.Adam:A method for stochastic optimization[C]//International Conference on Learning Representations.2015.
[31] BEIN B.Entropy[J].Best Practice & Research Clinical Anaesthesiology,2006,20(1):101-109.
[32] QU G H,ZHANG D L,YAN P F.Information measure for performance of image fusion[J].Electronics Letters,2002,38(7):313-315.
[33] HAN Y,CAI Y Z,CAO Y,et al.A new image fusion perfor-mance metric based on visual information fidelity[J].Information Fusion,2013,14(2):127-135.
[34] XYDEAS C S,PETROVIC V.Objective image fusion perfor-mance measure[J].Electronics Letters,2000,36(4):308-309.
[35] ESKICIOGLU A M,FISHER P S.Image quality measures and their performance[J].IEEE Transactions on Communications,1995,43(12):2959-2965.
[36] CUI G M,FENG H J,XU Z H,et al.Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition[J].Optics Communications,2015,341:199-209.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发