Abstract: Outdoor videos obtained under rainy weather cause visual quality degradation,which affects the processing effects of visual tasks such as object recognition and tracking.In order to enhance the quality of video and complete the effective recovery of the details in the motion video,many methods have been proposed in video rain removal.At this stage,most of the video rain removal methods based on convolutional neural networks employ single-frame enhancement and multi-frame fusion to remove rain.But the movement of some pixels between adjacent frames with direct enhancement is difficult to be completed in the temporal dimension.And the manner cannot effectively achieve end-to-end training,making the final result still relatively blurry and many detailed information losses.In order to effectively solve the above problems,this paper proposes a multi-frame fusion rain removal network based on the combination of motion estimation and space-temporal feature aggregation,ME-Derain for short.First,the optical flow estimation method is used to establish a reference frame to complete the alignment of adjacent frames,and then an encoder-decoder structure is introduced.The convolutional neural network connected by the residual connection and the time-related attention enhancement mechanism together form a multi-frame fusion network.Finally,the enhancement module related to the spatial sequence is used to obtain the rain removal video.A large number of experiments on different data sets show that the proposed method is better than most common methods at this stage and can obtain better rain removal effect.

Key words: Convolutional neural network, Optical flow, Video enhancement, Video rain removal