计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250600235-9.doi: 10.11896/jsjkx.250600235

• 图像处理&多媒体技术 • 上一篇    下一篇

基于条件化双网络的红外与可见光图像光照自适应融合

王熔硕, 王佳佳, 贾振红, 周刚   

  1. 新疆大学计算机科学与技术学院 乌鲁木齐 830017
    新疆维吾尔自治区信号检测与处理重点实验室 乌鲁木齐 830017
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 王佳佳(wjjxj@xju.edu.cn)
  • 作者简介:(rs_wang@stu.xju.edu.cn)
  • 基金资助:
    “天山英才”科技创新团队项目(2023TSYCTD0012)

Conditional Dual-network Fusion for Illumination-adaptive Infrared and Visible Image

WANG Rongshuo, WANG Jiajia, JIA Zhenhong, ZHOU Gang   

  1. School of Computer Science and Technology,Xinjiang University,Urumqi 830017,ChinaXinjiang Uygur Autonomous Region Signal Detection and Processing Key Laboratory,Xinjiang University,Urumqi 830017,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:WANG Rongshuo,born in 1998,postgra-duate.His main research interests include deep learning and image fusion.
    WANG Jiajia,Ph.D,professor,is a member of CCF(No.T1019M).Her main research interests include biosen-sing,signal and information processing.
  • Supported by:
    Tianshan Talent Training Project Xinjiang Science and Technology Innovation Team Program(2023TSYCTD0012).

摘要: 红外图像与可见光图像融合在复杂场景下能兼顾细节与目标,但昼夜光照差异导致两种图像的关注点存在内在冲突:白天应保留可见光图像的纹理结构,夜间则依赖红外图像突出目标,单一网络难以在不同光照条件下实现融合策略的最优权衡,导致性能退化。因此,研究目的是解决这种跨光照条件下的策略冲突,实现一种能够自适应光照变化的稳健融合方法。为此,提出一种条件化双网络框架:通过光照感知实现对昼夜场景的自适应分配,并在融合过程中设计互补信息提取与软切换机制,以平滑应对连续光照过渡。实验在 MSRS,M3FD 与 TNO 数据集上进行,结果表明,该方法在结构保真与目标显著性指标上均取得领先,显著克服了昼夜冲突带来的性能瓶颈。结果验证了光照自适应建模是提升红外图像与可见光图像融合鲁棒性的重要途径。

关键词: 可见光图像, 红外图像, 图像融合, 光照自适应, 双融合网络, 引导式互补信息提取

Abstract: Fusion of infrared and visible images can take into account both details and targets in complex scenes.Still,the diffe-rence in day and night illumination leads to an inherent conflict in the focus of the two types of images:during the daytime,the texture structure of the visible image should be preserved,while at night,it relies on the infrared image to highlight the target,and it is difficult for a single network to optimize under different illumination at the same time,which results in degradation of the performance.The research objective of this paper is to resolve the policy conflict across different illumination conditions and develop a robust fusion method that can adapt to changes in illumination.To this end,this paper proposes a conditionalized dual-network framework that adapts the assignment of day/night scenes through light sensing and designs complementary information extraction and soft switching mechanisms during the fusion process to cope with continuous light transitions smoothly.Experiments on the MSRS,M3FD,and TNO datasets demonstrate that the method outperforms in both structural fidelity and target saliency metrics,significantly alleviating the performance bottleneck caused by day-night conflicts.The results verify that light adaptive modelling is an important way to improve the robustness of the fusion of infrared and visible images.

Key words: Visible images, Infrared images, Image fusion, Light adaptation, Dual fusion network, Guided complementary information extraction

中图分类号: 

  • TP391
[1] LI M,HAO Y H,XU S Y.Research on fusion algorithm of infrared and visible light images[J].Fire and Command and Control,2025,50(3):165-177.
[2] LIU K,LI M,ZUO E,et al.ASFFuse:Infrared and visible image fusion model based on adaptive selection feature maps[J].Pattern Recognit,2024(149):110226.
[3] GAN W X,PAN J J,GENG J,et al.An infrared and visible image fusion method for all-weather road scenes[J].Journal of Wuhan University(Information Science Edition),2025,50(7):1346-1358.
[4] ZHU Z W,SONG X O,CUI W,et al.A review of visible-infrared image fusion for target detection[J].Computer Engineering and Applications,2025,61(17):17-32.
[5] CHEN H,LIN M,LIU J,et al.NT-DPTC:A non-negative temporal dimension preserved tensor completion model for missing traffic data imputation[J]. Information Science,2024(653):119797.
[6] LI S H,CAI W,WANG X,et al.A review of infrared and visibleimage fusion methods under deep learning framework[J].Computer Engineering and Applications,2025,61(9):25-40.
[7] LIU X,HUO H,YANG X,et al.A three-dimensional feature-based fusion strategy for infrared and visible image fusion[J].Pattern Recognition,2025,157:110885.
[8] LIU Y,CHEN X,CHENG J,et al.Infrared and visible image fusion with convolutional neural networks[J].Interational Journal of Wavelets,Multiresolution and Information Processing,2018,16(3):1-20.
[9] ZHANG H,XU H,XIAO Y,et al.Rethinking the image fusion:A fast unified image fusion network based on proportional maintenance of gradient and intensity[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:12797-12804.
[10] MA J,YU W,LIANG P,et al.FusionGAN:A generative adversarial network for infrared and visible image fusion[J].Information Fusion,2019,4:11-26.
[11] LI H,WU X J,DenseFuse:A fusion approach to infrared and visible images,IEEE Trans[J].Image Process,2019,28(5):2614-2623.
[12] HUANG G,LIU Z,VAN DER MAATEN L,et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4700-4708.
[13] TANG L,YUAN J,ZHANG H,et al.PIAFusion:A pro-gressive infrared and visible image fusion network based on illuminationaware[J].Information Fusion,2022,83-84:79-92.
[14] LIU J,FAN X,HUANG Z,et al.Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for objectdetection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recongnition.2022:5802-5811.
[15] SUN Y,DONG M,YU M,et al.HAIAFusion:A Hybrid Attention Illumination-Aware Framework for Infrared and Visible Image Fusion[J].IEEE Transactions on Instrumentation and Measurement,2025,74:5006619.
[16] LI H,WU X J.CrossFuse:A novel cross attention mechanism based infrared and visible image fusion approach[J].Information Fusion,2024,103:102147.
[17] CHEN J,YANG L,LIU W,et al.Lenfusion:a joint low-light enhancement and fusion network for nighttime infrared and visible image fusion[J].IEEE Transactions on Instrumentation and Measurement,2024,73:5018715.
[18] TANG W,HE F,LIU Y.ITFuse:An interactive transformer for infrared and visible image fusion[J].Pattern Recognition,2024,156:110822.
[19] SUN Y,DONG M,YU M,et al.MBHFuse:A multi-branch heterogeneous global and local infrared and visible image fusion with differential convolutional amplification features[J].Optics &Laser Technology,2025,181:111666.
[20] LI J,YU H,CHEN J,et al.A-RNet:Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:4770-4778.
[21] ROBERTS J W,VAN AARDT J A,AHMED F B.Assessment of image fusion procedures using entropy,image quality,and multispectral classification[J],Journal of Applied Remote Sen-sing,2008,2(1):023522.
[22] ESKICIOGLU A M,FISHER P S.Image quality measures and their performance[J].IEEE Transactions on Communications,1995,43(12):2959-2965.
[23] WANG Z,SIMONCELLI E P,BOVIK A C.Multiscale structu-ral similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals,Systems & Compu-ters.2003.
[24] HAN Y,CAI Y,CAO Y,et al.A new image fusion performance metric based on visual information fidelity[J].Information Fusion,2013,14(2):127-135.
[25] XYDEAS C S,PETROVIC V.Objective imagefusion perfor-mance measure [J].Electronics Letters,2000,36(4):308-309.
[26] REDMON J,DIVVALA S,GIRSHICKR,et al.You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:779-788.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!