Abstract: Video caption detection and extraction is one of the key technologies forvideo understanding.This paper proposed a two-stage approach which divides the process into caption frame and caption area,improving the caption detection efficiency and accuracy.In the first stage,caption frame detection and extraction is conducted.Firstly,the motion detection is performed according to the gray correlation frame difference,the captions are judged initially,and a new binary image sequence is obtained.Then,according to dynamic characteristics of ordinary captions and scrolling captions,the new sequence is screened two times to get caption frame.In the second stage,caption area detection and extraction is conducted.Firstly,the Sobel edge detection algorithm is used to detect the caption region,and the background is eliminated according to the constraint height.Then according to the aspect ratio,the vertical and horizontal captions are distinguished,and all captions in the caption frame can be obtained,including static captions,ordinary captions and scrol-ling captions.This method reduces the frames which need to be detected and improves caption detection efficiency by 11%.The experimental results show that the proposed method can approximately improve the F score by 9% compared with the methods of separately using the gray correlation frame difference and edge detection.

Key words: Detection and extraction, Dynamic characteristics, Gray correlation frame difference, Sobel edge detection, Video caption