关键词: 图像篡改检测, 多模态融合, 模态不平衡, 协同融合, 图像处理

Abstract: To address the challenges of increasingly diverse and covert manipulation techniques in the field of image tampering detection,existing methods have attempted to fuse auxiliary modal information to enhance detection capabilities.However,they often only consider fusing a single auxiliary modality,which limits the comprehensive capture of diverse tampering clues and thereby suppresses overall fusion performance.To solve this key challenge,firstly,a multi-stage synergistic fusion network named MSBF-Net is proposed.This network adopts a Transformer-based encoder-decoder architecture capable of effectively processing feature information at different hierarchical levels.Secondly,a multi-branch modality-cooperative encoder is designed to jointly model RGB visual features with three complementary noise fingerprints－DCT,SRM,and NoisePrint++－to capture rich forgery clues from multiple dimensions and enhance the representation capability for diverse manipulations.Thirdly,to achieve efficient inter-action between the feature branches,a tri-source attention adaptive fusion module is designed.It deeply integrates the multi-path feature streams at different stages of the encoder,strengthening the complementary information between modalities through an adaptive attention mechanism.Finally,to solve the critical problem of modal imbalance in multi-modal learning and to improve localization precision,a mechanism based on class prototypes,comprising prototypical class entropy(PCE) loss and prototypical entropy regularization(PER),is designed to guide the balanced development of each modality.The effectiveness of this method is verified through comprehensive experiments on five public datasets,with the results demonstrating the comprehensive perfor-mance advantages of the MSBF-Net model.Its advanced nature is particularly evident when tackling the CocoGlide dataset,which is generated by advanced diffusion models.On this dataset,the model achieves state-of-the-art results in the two critical tasks of pixel-level localization and image-level detection,with performance improvements of approximately 3.82% and 5.95%,respectively,compared to the previous state-of-the-art model.Simultaneously,on multiple benchmark datasets featuring traditional manipulation types such as splicing and copy-move,the model’s core metrics also consistently remain in the top tier,with its average metrics being optimal,which verifies its strong generalization ability.Furthermore,ablation studies and robustness tests further confirm that the model’s internal components form an effective synergy and that its performance is stable under various image distortion conditions.

Key words: Image manipulation detection, Multi-modal fusion, Modal imbalance, Synergistic fusion, Image processing