Abstract: Considering the small motion in image sequence,this paper proposed a line tracking and matching algorithm based on semi-direct method is proposed.Firstly,extracting and matching of feature point and line should be conducted in the key frames.Secondly,feature point of the line is reconstructed by using the method of structure from motion.Then,feature point tracking and relative pose estimation are calculated through inverse compositional image alignment algorithm.Finally,line matching result is obtained based on feature point tracking result.Two group of image sequence experiments were conducted to validate the proposed algorithm.The experiments results indicate that the proposed algorithm is capable of tracking and matching the lines in the image sequence,and can estimate the camera pose simultaneously.And camera pose error accumulates with the increase of image frame.A novel line matching algorithm in image sequence was proposed.The algorithm can achieve line tracking and matching and obtain camera track at the same time through sparse feature point of the line.However,the algorithm needs to be corrected because of accumulate error.

Key words: Image sequence, Line tracking and matching, Relative pose estimation, Semi-direct method