Computer Science ›› 2019, Vol. 46 ›› Issue (7): 286-291.doi: 10.11896/j.issn.1002-137X.2019.07.044

• Graphics, Image & Pattern Recognition • Previous Articles     Next Articles

Video Stitching Algorithm Based on SIFT and Its Optimization

YANG Si-yan1,HE Guo-qi1,LIU Ru-yi2   

  1. (School of Information and Intelligence Technology,Shaanxi Radio & TV University,Xi’an 710119,China)1
    (School of Computer Science and Technology,Xidian University,Xi’an 710071,China)2
  • Received:2018-07-25 Online:2019-07-15 Published:2019-07-15

Abstract: At present,a large number of video information obtained by independent video devices in small scenes cannot meet the requirements of information processing in large scenes,and the manual access by multiple devices is faced with such problems as low efficiency,information redundancy and fragmentation.This paper investigated large scene video stitching technology,carried out the feature matching of key points by using the scale-invariant feature of SIFT algorithm,and completed the image splicing by affine matrix deformation in this process.The traditional SIFT splicing algorithm is further optimized,mainly based on distance optimization algorithm to improve the effect of video splicing.In order to improve the efficiency of stitching,the techniques of key frame extraction based on SIFT feature point matching weighted optimization algorithm were accelerated in parallel.Experiments show that the proposed optimization method can extract essential information in the video more effectively,achieving better video stitching results.Visually,the intersection of the two images appeared in the results of the conventional method does not appear in the stitching results obtained by the proposed method.Moreover,the key point detection and splicing parts were accelerated and optimized respectively in the different environment.In MATLAB,the efficiency of the key point with optimized is improved by 20%,and that of splicing parts is increased by about 57%.In C++,the efficiency of the key point is improved by 14%,and that of splicing parts is increased by about 40%.

Key words: Key frame, Parallel optimization, SIFT feature point matching, Video Stitching, Weighted optimization algorithm

CLC Number: 

  • TP391
[1]ZHANG J,CHEN G,JIA Z.An image stitching algorithm based on histogram matching and SIFT algorithm [J].International Journal of Pattern Recognition and Artificial Intelligence,2017,31(4):1754006.
[2]ZHENG G L,TANG B B.The stereo vision positioning system based on multiple fisheye cameras [J].Computer Simulation,2016,33(7):256-260.(in Chinese)
郑贵林,唐贝贝.基于多鱼眼摄像头的立体视觉定位系统[J].计算机仿真,2016,33(7):256-260.
[3]KAUR S,ER S K.Analysis of Image Stitching for Noisy Images using SIFT [J].International Journal of Advanced Research in Computer Science,2017,8(5):2078-2082.
[4]WANG Y W,YU M,JIANG H,et al.An image stitching algorithm via adaptive quadtree segmentation[J].Journal of Ningbo University (Natural Science & Engineering),2018,31(4):52-58.(in Chinese)
王元炜,郁梅,姜浩,等.一种自适应四叉树分块的图像拼接算法[J].宁波大学学报(理工版),2018,31(4):52-58.
[5]HE C,ZHOU J.Mesh-based image stitching algorithm with li- near structure protection[J].Journal of Image and Graphics,2018,23(7):0973-0983.(in Chinese)
何川,周军.具有直线结构保护的网格化图像拼接[J].中国图象图形学报,2018,23(7):973-983.
[6]ZHOU X,CAO S,HE X J,et al.Image stitching based on the planar similarity among matching pairs of feature points [J].Journal of University Electronic Science and Technology of China,2017,46(6):877-882.(in Chinese)
周雪,曹爽,何香静,等.基于特征点匹配对平面相似度的图像拼接[J].电子科技大学学报,2017,46(6):877-882.
[7]WANG F B,TU P,WU C,et al.Multi-image mosaic with SIFT and vision measurement for microscale structures processed by femtosecond laser [J].Optics and Lasers in Engineering,2018,100:124-130.
[8]FATHIMA A A,KARTHIK R,VAIDEHI V.Image stitching with combined moment invariants and sift features [J].Procedia Computer Science,2013,19:420-427.
[9]LIU W L,WANG Z Y,QING L B,et al.Application of LBP algorithm in rock slice image stitching[J].Computer & Digital Engineering,2016,44(2):326-330.(in Chinese)
刘文亮,王正勇,卿粼波,等.LBP 算法在岩石薄片图像拼接中的应用[J].计算机与数字工程,2016,44(2):326-330.
[10]CHEN Y,ZHAO Y,WANG S G.Fast image stitching method based on SIFT feature vector image [J].Journal of Jilin University (Science Edition),2017,5(1):116-122.(in Chinese)
陈月,赵岩,王世刚.基于 SIFT 特征矢量图的快速图像拼接方法[J].吉林大学学报(理学版),2017,55(1):116-122.
[11]LU J M,ZHU Z.Real-time 4K panoramic video stitching based on GPU acceleration [J].Computer Science,2017,44(8):18-21.(in Chinese)
卢嘉铭,朱哲.基于 GPU 加速的实时 4K 全景视频拼接[J].计算机科学,2017,44(8):18-21.
[12]LU Y Y,ZHANG M.Improved Algorithm Based on SIFT Infrared Image Stitching Algorithm [J].Computer Systems & Applications,2015,24(8):165-170.(in Chinese)
陆园园,张明.基于 SIFT 算法的红外图像拼接方法改进[J].计算机系统应用,2015,24(8):165-170.
[13]CHEN Y,ZHAO Y,WANG S G.Fast image stitching method based on SIFT with adaptive local image feature[J].Chinese Optics,2016,9(4):415-422.(in Chinese)
陈月,赵岩,王世刚.图像局部特征自适应的快速SIFT图像拼接方法[J].中国光学,2016,9(4):415-422.
[14]ZHAO Y,CHEN Y,WANG S G.Corrected fast SIFT image stitching method by combining projection error [J].Optics and Precision Engineering,2017,25(6):1645-1651.(in Chinese)
赵岩,陈月,王世刚.结合投影误差校正的快速 SIFT 图像拼接[J].光学精密工程,2017,25(6):1645-1651.
[15]XU W,MULLIGAN J.Performance evaluation of color correction approaches for automatic multi-view image and video stitching[C]∥2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE,2010:263-270.
[16]DU C,YUAN J,DONG J,et al.GPU based Parallel Optimization for Real Time Panoramic Video Stitching [J].arXiv1810.03988,2018.
[17]JIANG W,GU J.Video stitching with spatial-temporal content-preserving warping[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.IEEE,2015:42-48.
[18]LIU J,LI S Y,LI R F.Multi-view Video Stitching of Outdoor Scenes [J].Computer Engineering,2016,42(4):259-265.(in Chinese)
刘娟,李实英,李仁发.室外场景的多视角视频拼接[J].计算机工程,2016,42(4):259-265.
[19]LI Y,DU B X.Real-time video splicing technology based on small region fusion [J].Journal of Jilin University (Science Edition),2016,54(6):1367-1372.(in Chinese)
李勇,杜丙新.基于小区域融合的实时视频拼接技术[J].吉林大学学报 (理学版),2016,54(6):1367-1372.
[20]YANG X P,HU Y,ZHANG K.Research on video mosaicking technology based on FPGA [J].Journal of Jilin University (Information Science Edition),2016,34(6):709-715.(in Chinese)
杨晓萍,胡玉,张凯.基于 FPGA 的视频拼接技术研究[J].吉林大学学报(信息科学版),2016,34(6):709-715.
[21]CHANG J Y,QIN R,LI Q,et al.Image quality assessment of panoramic image[J].Computer Science,2014,41(6):278-281.(in Chinese)
常嘉义,秦瑞,李庆,等.全景鸟瞰拼接图像的质量评价方法[J].计算机科学,2014,41(6):278-281.
[22]NOWOZIN S.Autopano-Sift,making panoramas fun[OL].ht- tp://user.cs.tu-berlin.De/nowozin/autopano-sift.
[23]PERBET F, JOHNSON S,PHAM M T,et al.Human Body Shape Estimation Using a Multi-resolution Manifold Forest[C]∥2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Columbus,OH,USA,2014:668-675.
[24]WANG J,WANG J D,ZENG G,et al.Fast Neighborhood Graph Search Using Cartesian Concatenation[C]∥2013 IEEE International Conference on Computer Vision (ICCV).Sydney,NSW,Australia,2013:2128-2135.
[1] LIU Yun, DONG Shou-jie. Acceleration Algorithm of Multi-channel Video Image Stitching Based on CUDA Kernel Function [J]. Computer Science, 2022, 49(6A): 441-446.
[2] LV Xiao-jing, LIU Zhao, CHU Xue-sen, SHI Shu-peng, MENG Hong-song, HUANG Zhen-chun. Extreme-scale Simulation Based LBM Computing Fluid Dynamics Simulations [J]. Computer Science, 2020, 47(4): 13-17.
[3] WEI Lin-jing, NING Lu-lu, GUO Bin, HOU Zhen-xing, GAN Shi-run. K-mediods Cluster Mining and Parallel Optimization Based on Shuffled Frog Leaping Algorithm [J]. Computer Science, 2020, 47(10): 126-129.
[4] HU Zhi-jun,XU Yong. Overview of Content-based Video Retrieval [J]. Computer Science, 2020, 47(1): 117-123.
[5] LIU Yu-cheng, Richard·DING, ZHANG Ying-chao. Research on Pan-real-time Problem of Medical Detection Based on BPNNs Recognition Algorithm [J]. Computer Science, 2018, 45(6): 301-307.
[6] JIN Xin, LI Long-wei, SU Guo-hua, LIU Xiao-lei and JI Jia-nan. Prediction about Network Security Situation of Electric Power Telecommunication Based on Spark Framework and PSO Algorithm [J]. Computer Science, 2017, 44(Z6): 366-371.
[7] GUO Xin-peng, HUANG Yuan-yuan and HU Zuo-jin. Research on Continuous Sign Language Sentence Recognition Algorithm Based on Key Frame [J]. Computer Science, 2017, 44(Z11): 178-183.
[8] LIANG Wen-le, HUANG Yuan-yuan and HU Zuo-jin. Real-time Dynamic Sign Language Recognition Based on Hierarchical Matching Strategy [J]. Computer Science, 2017, 44(7): 299-303.
[9] JIANG Wen-chao, LIN Sui, WANG Duo-qiang, LI Dong-ming and JIN Hai. Three-level Parallel Optimization and Application of Calculix in TH-2 Super-computing Environments [J]. Computer Science, 2017, 44(3): 32-35.
[10] ZHONG Xian, YANG Guang and LU Yan-sheng. Method of Key Frames Extraction Based on Double-threshold Values Sliding Window Sub-shot Segmentation and Fully Connected Graph [J]. Computer Science, 2016, 43(6): 289-293.
[11] JIANG Yong and ZHANG Hai-tao. Representative Selection Framework Approach for Videos [J]. Computer Science, 2016, 43(11): 19-23.
[12] LIU Hua-yong and LI Tao. Key Frame Extraction Algorithm Based on Improved Block Color Features and Second Extraction [J]. Computer Science, 2015, 42(12): 307-311.
[13] . Content-based Video Forensics [J]. Computer Science, 2012, 39(1): 27-31.
[14] ZHU Zheng-Zhou, WU Zhong-Fu, WANG Qian (College of Computer Science and Technology, Chongqing University, Chongqing 400044). [J]. Computer Science, 2008, 35(2): 129-131.
[15] RONG Jia-Wei, WU Li-De (Fudan University, Shanghai 200433). [J]. Computer Science, 2005, 32(12): 220-222.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!