Abstract: Event-based camera is commonly seen in object detection in limited scenarios for traditional camera applications (high speed,strong light,low light,etc.) due to their high time resolution,high dynamic range and low power consumption.However,the event sequence output of event camera is difficult to be manually labeled due to its pixel asynchronism,so the existing me-thods obtain event sequence annotations through the migration of RGB image annotations.However,since the migrated annotations have numerous inaccurate bounding boxes and some object textures in event sequence are fuzzy,leading to poor model performance.To address this problem,event-based camera object detection algorithm for cross-modal noisy annotations filtering is proposed.The method uses a pre-trained event-based camera detector to filter open-source RGB object detection datasets and selects RGB images that are most valuable for training the event-based camera detector.These selected RGB images are combined with event images to construct cross-domain mixed images,helping the detector to identify and locate the event image object more accurately.To mitigate the impact of noisy annotations on detector performance,a multi-stage object detection joint optimization strategy is designed.After each stage of training is completed,noisy annotations are identified in the global annotations and are corrected use in the next stage.Experimental results show that,on the 1Mpx Detection Dataset,the robust event-based camera cross-modal object detection method based on noisy annotations provides 8.35% model gain compared to the baseline model,significantly outperforming noise-label learning methods such as Co-teaching and O2U-net.Specifically,cross-modal hybrid images training and joint optimization frameworks offer model gains of 6.44% and 4.77%,respectively.

Key words: Event-based camera, Object detection, Noisy annotations, Cross-modal, Joint optimization