计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (2): 212-221.doi: 10.11896/jsjkx.231200068

• 计算机图形学&多媒体 • 上一篇    下一篇

基于改进网格优化的大视野双目图像拼接算法

董辉, 张渊淞, 林文杰, 吴祥, 郭方洪, 张丹   

  1. 浙江工业大学信息工程学院 杭州 310023
  • 收稿日期:2023-12-10 修回日期:2024-05-16 出版日期:2025-02-15 发布日期:2025-02-17
  • 通讯作者: 吴祥(xiangwu@zjut.edu.cn)
  • 作者简介:(hdong@zjut.edu.cn)
  • 基金资助:
    国家自然科学基金(62203391)

Improved Mesh Optimization-based Image Stitching Algorithm of Large Field Binocular Vision

DONG Hui, ZHANG Yuansong, LIN Wenjie, WU Xiang, GUO Fanghong, ZHANG Dan   

  1. College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2023-12-10 Revised:2024-05-16 Online:2025-02-15 Published:2025-02-17
  • About author:DONG Hui,born in 1979,Ph.D,professor,Ph.D supervisor.His main research interests include robotics,industrial Internet of Things,and embedded system technology and applications.
    WU Xiang,born in 1990,Ph.D,associate researcher.His main research interests include networked control system and intelligent optimization algorithms.
  • Supported by:
    National Natural Science Foundation of China(62203391).

PDF (PC)

121

摘要： 针对双目图像拼接中的色彩不一致、失真、伪影等造成拼接质量较差的问题,提出了基于改进网格优化的大视野双目图像拼接算法。首先,设计了基于平移变换和全局对准约束的改进网格优化方法,通过最小化由点线对准项、全局对准项和显著线保持项构成的目标函数,得到最优网格顶点集,在实现图像配准的同时保持原始的形状结构信息。其次,设计了改进直方图匹配和接缝搜索的图像融合算法,通过改进直方图匹配消除了重叠区域亮度色调的差异以及大视差工况下的偏色现象。采用基于人眼感知的接缝搜索方法获得缝合线后加权融合,可有效避免在未对齐的特征稀疏区域使用加权融合而导致的伪影。最后,在10个大视野场景下将所提算法与SPW、LPC、PSC和联咏这4种算法进行对比实验,所提算法相较于性能次优算法的平均配准误差减小了28.1%,平均失真误差减小了99.5%。结果表明,所提算法不仅能有效消除双目图像之间的色调差异,而且可抑制目标图像非重叠区域的投影畸变,以及很好地去除重叠区域的伪影,具有明显的优越性。

关键词: 双目视觉, 网格优化, 图像配准, 图像拼接, 色彩校正, 最佳缝合线

Abstract: To solve the problems of poor stitch quality caused by color inconsistency,distortion and artifacts in binocular image stitch,a large field binocular image stitch algorithm based on improved mesh optimization is proposed.Firstly,an improved mesh optimization method based on translation transformation and global alignment constraints is designed.By minimizing an objective function consisting of point-line alignment term,global alignment term and salient line preservation term,the optimal mesh vertex set is obtained,which achieves image registration while preserving the original shape and structure information.Secondly,an image fusion algorithm based on improved histogram matching and seam search is designed.The difference of brightness and hue in the overlapping area and the color cast phenomenon under large disparity are eliminated by improved histogram matching,and the seam search method based on human perception is used to obtain the weighted fusion after the suture line,which can effectively avoid the artifacts caused by using weighted fusion in the unaligned feature sparse area.Finally,the proposed algorithm is compared with four algorithms such as SPW,LPC,PSC and NOVATEK in 10 large-field scenes.The average registration error of the proposed algorithm is reduced by 28.1% and the average distortion error is reduced by 99.5% compared with the suboptimal algorithm.The results show that the proposed algorithm can not only effectively eliminate the hue difference between binocular images,but also suppress the projection distortion in the non-overlapping area of the target image and remove the artifacts in the overlapping area,which has obvious advantages.

Key words: Binocular vision, Grid optimization, Image registration, Image stitching, Color correction, Optimal seam search

中图分类号: 

  • TP399
[1]ZHENG J,WANG Y,WANG H,et al.A novel projective-consistent plane based image stitching method[J].IEEE Transactions on Multimedia,2019,21(10):2561-2575.
[2]WAN Q,CHEN J,LUO L,et al.Drone image stitching using local mesh-based bundle adjustment and shape-Preserving transform[J].IEEE Transactions on Geoscience and Remote Sen-sing,2020,59(8):7027-7037.
[3]XIA D,ZHOU R.Survey of Parallax Image Registration Technology [J].Journal of Computer Engineering & Applications,2021,57(2):18-27.
[4]YAN X Y,LU F F,GE L S,et al.Image Style Transfer Based on the Distribution Matching of the Style Features[J].Journal of Chongqing Technology and Business University(Natural Science Edition),2023,40(3):48-55.
[5]FU M,LIANG H,ZHU C,et al.Image Stitching TechniquesApplied to Plane or 3D Models:A Review[J].IEEE Sensors Journal,2023,23(8):8060-8079.
[6]KAUR H,KOUNDAL D,KADYAN V.Image fusion tech-niques:a survey[J].Archives of computational methods in Engineering,2021,28:4425-4447.
[7]NIE L,LIN C,LIAO K,et al.A view-free image stitching network based on global homography[J].Journal of Visual Communication and Image Representation,2020,73:102950.
[8]SONG D Y,LEE G,LEE H K,et al.Weakly-Supervised Sti-tching Network for Real-World Panoramic Image Generation[C]//European Conference on Computer Vision.Cham:Sprin-ger Nature Switzerland,2022:54-71.
[9]NIE L,LIN C,LIAO K,et al.Unsupervised deep image sti-tching:Reconstructing stitched features to images[J].IEEE Transactions on Image Processing,2021,30:6184-6197.
[10]LIU F,GLEICHER M,JIN H,et al.Content-preserving warps for 3D video stabilization[J].ACM Transactions on Graphics,2009,28(3):1-9.
[11]ZARAGOZA J,CHIN T J,BROWN M S,et al.As-projective-as-possible image stitching with moving DLT[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:2339-2346.
[12]LI S,YUAN L,SUN J,et al.Dual-feature warping-based motion model estimation[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:4283-4291.
[13]JOO K,KIM N,OH T H,et al.Line meets as-projective-as-possible image stitching with moving DLT[C]//2015 IEEE International Conference on Image Processing(ICIP).IEEE,2015:1175-1179.
[14]DU P,NING J,CUI J,et al.Geometric Structure PreservingWarp for Natural Image Stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:3688-3696.
[15]YUAN X,ZHENG Y,ZHAO W,et al.Image stitching method by multi-feature constrained alignment and colour adjustment[J].IET Image Processing,2021,15(7):1499-1507.
[16]LIAO T,CHEN J,XU Y.Quality evaluation-based iterativeseam estimation for image stitching[J].Signal,Image and Video Processing,2019,13:1199-1206.
[17]ZHANG J,GAO Y,XU Y,et al.A simple yet effective image stitching with computational suture zone[J].The Visual Computer,2023,39(10):4915-4928.
[18]CHEN X,YU M,SONG Y.Optimized seam-driven image sti-tching method based on scene depth information[J].Electronics,2022,11(12):1876.
[19]ZHANG J,WANG C,LIU S,et al.Content-aware unsupervised deep homography estimation[C]//Computer Vision－ECCV 2020:16th European Conference,Glasgow,UK,August 23－28,2020,Proceedings,Part I 16.Springer International Publishing,2020:653-669.
[20]SHEN X,DARMON F,EFROS A A,et al.Ransac-flow:generic two-stage image alignment[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23－28,2020,Proceedings,Part IV 16.Springer International Publishing,2020:618-637.
[21]LI A,GUO J,GUO Y.Image stitching based on semantic planar region consensus[J].IEEE Transactions on Image Processing,2021,30:5545-5558.
[22]NIE L,LIN C,LIAO K,et al.Deep rectangling for image stitching:a learning baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5740-5748.
[23]XU X Y,YUAN S S,WANG J,et al.Image stitching method based on global and local features [J].Journal of Beijing Institute of Technology,2022,42(5):502-510.
[24]ALCANTARILLA P F,BARTOLI A,DAVISON A J.KAZE features[C]//Computer Vision－ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,October 7－13,2012,Proceedings,Part VI 12.Springer Berlin Heidelberg,2012:214-227.
[25]CHIN T J,YU J,SUTER D.Accelerated hypothesis generation for multistructure data via preference analysis[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2011,34(4):625-638.
[26]DING C,MA Z.Multi-camera color correction via hybrid histogram matching[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(9):3327-3337.
[27]OTSU N.A threshold selection method from gray-level histograms[J].IEEE Transactions on Systems,Man,and Cyberne-tics,1979,9(1):62-66.
[28]LIAO T,LI N.Single-perspective warps in natural image sti-tching[J].IEEE Transactions on Image Processing,2019,29:724-735.
[29]JIA Q,LI Z J,FAN X,et al.Leveraging line-point consistence to preserve structures for wide parallax image stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:12186-12195.
[30]LI N,LIAO T,WANG C.Perception-based seam cutting forimage stitching[J].Signal,Image and Video Processing,2018,12:967-974.
[31]VON GIOI R G,JAKUBOWICZ J,MOREL J M,et al.LSD:A line segment detector[J].Image Processing On Line,2012,2:35-55.
[32]NIE L,LIN C,LIAO K,et al.Parallax-Tolerant UnsupervisedDeep Image Stitching[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:7399-7408.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发