Abstract: To address the difficulty of self-encoder image fusion algorithms in highlighting infrared(IR) salient targets and the challenge of simultaneously considering global structure and local detail information in existing fusion strategies－while most algorithms overly prioritize statistical metrics and overlook support for advanced visual tasks－a semantic segmentation-guided image fusion method with a hybrid cross-feature mechanism is proposed.Shallow and deep skip connections are introduced between the encoder and decoder,employing a maximum value selection strategy to emphasize salient targets and reduce redundancy.The fusion strategy integrates global context and local fine-grained information through cross-attention and convolutional operations,combining different modal features within a single frame.The fused image is then fed into a segmentation network,where semantic loss guides high-level semantic information back to the fusion network,enabling the generation of a fused image rich in semantic detail.Experimental results demonstrate that the proposed method achieves average improvements of 33.93%,112.81%,49.89%,27.64%,and 23.87% in SD,MI,VIFF,Qabf,and AG metrics on the RoadScene dataset compared to seven baseline algorithms.Additionally,the intersection and concurrency ratios for car,person,and bicycle categories in the semantic segmentation task on the MSRS dataset increase by 3.47%,6.37%,and 9.57% on average,outperforming other state-of-the-art methods.

Key words: Image fusion, Infrared and visible image, Cross attention mechanism, Convolution, Semantic segmentation