Abstract: Visible and thermal(RGBT) salient object detection(SOD) aims to identify common salient objects from RGB and thermal infrared images.However,existing methods are predominantly trained on well-aligned image pairs,overlooking the “weak alignment” issue caused by sensor discrepancies during actual imaging.This issue refers to the fact that the same object,while structurally relates in different modalities,exhibits differences in position and scale.Therefore,training models with weakly-aligned RGBT images without alignment processing will lead to a significant reduction in detection performance.To address this challenge,a multi-modal alignment and fusion network(AFNet) is specifically designed for weakly-aligned RGBT SOD.The network comprises three main modules:the distribution alignment module(DAM),the attention-guided deformable convolution alignment module(AGDCM),and the cross-attention fusion module(CAM).DAM is based on optimal transport theory,aims to make the distribution of thermal infrared and RGB features as close as possible,achieving initial feature alignment.AGDCM utilizes deformable convolution and incorporates attention weights in the process of learning feature offsets,allowing different regions to learn suitable offsets for themselves,thereby achieving precise multi-modal feature alignment.CAM employs a cross-attention mechanism to fuse the aligned features,enhancing the discriminative capability of the fused features and improving computational efficiency.Extensive experiments on both aligned and weakly-aligned datasets demonstrate the effectiveness of the proposed method.

Key words: Weakly-aligned RGBT image, Salient object detection, Multi-modal feature alignment, Multi-modal feature fusion, Attention mechanism