Abstract: Automated tomato harvesting plays a pivotal role in enhancing agricultural efficiency and ensuring produce quality,but faces significant challenges in complex orchard environments where single-modal systems often fall short.Variabilities in lighting,occlusions by foliage,and the subtle characteristics of small targets in RGB images,coupled with inadequate feature extraction from single-sensor data,significantly impede precise detection.This study introduces the robust tomato detection with multi-modal fusion(RTDMF) model,designed to address these limitations by integrating RGB and infrared(IR) imaging technologies to bolster detection robustness.Constructed on the YOLOv5 framework,RTDMF incorporates lightweight depthwise separable convolutions and adaptive anchor boxes to enhance sensitivity towards small targets.The dual-branch architecture of RTDMF processes RGB and IR data independently,fusing color,texture,and thermal features effectively through self-attention mechanisms and specialized fusion modules.Furthermore,Mosaic data augmentation and dynamic learning rate strategies are employed to further enhance the model's generalization and convergence capabilities.Evaluated on a multimodal tomato dataset encompassing varying levels of maturity,diverse lighting conditions,and occlusion scenarios,RTDMF demonstrates a notable improvement,it achieves a 9.7% increase in mean average precision(mAP) and a0.6% higher recall rate compared to single-modal models.It also significantly reduces the miss and false detection rates by 2.3% and 3.1%,respectively.Visual analysis confirms the model's effectiveness in low-contrast and heavily occluded scenarios,showcasing its superior adaptability to real-world agricultural challenges.This multi-modal approach delivers a robust solution for automated harvesting systems in dynamic environments,marking a significant advancement in the field of agricultural automation.

Key words: Tomato small object detection, Multi-modal fusion, Adaptive anchor box optimization, Dual-branch architecture, Target feature refinement