Abstract: Infrared and visible image fusion aims to retain the thermal radiation information from infrared images and the texture details from visible images to represent the imaging scene and comprehensively promote downstream visual tasks.Fusion models based on convolutional neural networks(CNNs) encounter limitations in capturing global image features due to their focus on local convolutional operations.Although Transformer-based models excel in global feature modeling,they also face computational challenges posed by quadratic complexity.Recently,the selective structured state-space model(Mamba) has shown great potential in modeling long-range dependencies with linear complexity,providing a promising path to address the aforementioned issues.To efficiently model long-range dependencies in images,this paper designs a residual selective structured state space module(RMB) for extracting global features.Simultaneously,to model the relationship between multimodal images,a cross-modal query fusion attention module(CQAM) is designed for adaptive feature fusion.Furthermore,a loss function consisting of two terms,including gradient loss and brightness loss,is designed to train the proposed model in an unsupervised manner.Comparative experiments on fusion quality and efficiency with numerous other state-of-the-art methods and ablation studies demonstrate the effectiveness of the proposed MTFuse method.

Key words: Selective structured state space model, Transformer, Unsupervised learning, Infrared and visible image fusion