Computer Science ›› 2025, Vol. 52 ›› Issue (2): 212-221.doi: 10.11896/jsjkx.231200068

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Improved Mesh Optimization-based Image Stitching Algorithm of Large Field Binocular Vision

DONG Hui, ZHANG Yuansong, LIN Wenjie, WU Xiang, GUO Fanghong, ZHANG Dan   

  1. College of Information Engineering,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2023-12-10 Revised:2024-05-16 Online:2025-02-15 Published:2025-02-17
  • About author:DONG Hui,born in 1979,Ph.D,professor,Ph.D supervisor.His main research interests include robotics,industrial Internet of Things,and embedded system technology and applications.
    WU Xiang,born in 1990,Ph.D,associate researcher.His main research interests include networked control system and intelligent optimization algorithms.
  • Supported by:
    National Natural Science Foundation of China(62203391).

Abstract: To solve the problems of poor stitch quality caused by color inconsistency,distortion and artifacts in binocular image stitch,a large field binocular image stitch algorithm based on improved mesh optimization is proposed.Firstly,an improved mesh optimization method based on translation transformation and global alignment constraints is designed.By minimizing an objective function consisting of point-line alignment term,global alignment term and salient line preservation term,the optimal mesh vertex set is obtained,which achieves image registration while preserving the original shape and structure information.Secondly,an image fusion algorithm based on improved histogram matching and seam search is designed.The difference of brightness and hue in the overlapping area and the color cast phenomenon under large disparity are eliminated by improved histogram matching,and the seam search method based on human perception is used to obtain the weighted fusion after the suture line,which can effectively avoid the artifacts caused by using weighted fusion in the unaligned feature sparse area.Finally,the proposed algorithm is compared with four algorithms such as SPW,LPC,PSC and NOVATEK in 10 large-field scenes.The average registration error of the proposed algorithm is reduced by 28.1% and the average distortion error is reduced by 99.5% compared with the suboptimal algorithm.The results show that the proposed algorithm can not only effectively eliminate the hue difference between binocular images,but also suppress the projection distortion in the non-overlapping area of the target image and remove the artifacts in the overlapping area,which has obvious advantages.

Key words: Binocular vision, Grid optimization, Image registration, Image stitching, Color correction, Optimal seam search

CLC Number: 

  • TP399
[1]ZHENG J,WANG Y,WANG H,et al.A novel projective-consistent plane based image stitching method[J].IEEE Transactions on Multimedia,2019,21(10):2561-2575.
[2]WAN Q,CHEN J,LUO L,et al.Drone image stitching using local mesh-based bundle adjustment and shape-Preserving transform[J].IEEE Transactions on Geoscience and Remote Sen-sing,2020,59(8):7027-7037.
[3]XIA D,ZHOU R.Survey of Parallax Image Registration Technology [J].Journal of Computer Engineering & Applications,2021,57(2):18-27.
[4]YAN X Y,LU F F,GE L S,et al.Image Style Transfer Based on the Distribution Matching of the Style Features[J].Journal of Chongqing Technology and Business University(Natural Science Edition),2023,40(3):48-55.
[5]FU M,LIANG H,ZHU C,et al.Image Stitching TechniquesApplied to Plane or 3D Models:A Review[J].IEEE Sensors Journal,2023,23(8):8060-8079.
[6]KAUR H,KOUNDAL D,KADYAN V.Image fusion tech-niques:a survey[J].Archives of computational methods in Engineering,2021,28:4425-4447.
[7]NIE L,LIN C,LIAO K,et al.A view-free image stitching network based on global homography[J].Journal of Visual Communication and Image Representation,2020,73:102950.
[8]SONG D Y,LEE G,LEE H K,et al.Weakly-Supervised Sti-tching Network for Real-World Panoramic Image Generation[C]//European Conference on Computer Vision.Cham:Sprin-ger Nature Switzerland,2022:54-71.
[9]NIE L,LIN C,LIAO K,et al.Unsupervised deep image sti-tching:Reconstructing stitched features to images[J].IEEE Transactions on Image Processing,2021,30:6184-6197.
[10]LIU F,GLEICHER M,JIN H,et al.Content-preserving warps for 3D video stabilization[J].ACM Transactions on Graphics,2009,28(3):1-9.
[11]ZARAGOZA J,CHIN T J,BROWN M S,et al.As-projective-as-possible image stitching with moving DLT[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2013:2339-2346.
[12]LI S,YUAN L,SUN J,et al.Dual-feature warping-based motion model estimation[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:4283-4291.
[13]JOO K,KIM N,OH T H,et al.Line meets as-projective-as-possible image stitching with moving DLT[C]//2015 IEEE International Conference on Image Processing(ICIP).IEEE,2015:1175-1179.
[14]DU P,NING J,CUI J,et al.Geometric Structure PreservingWarp for Natural Image Stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:3688-3696.
[15]YUAN X,ZHENG Y,ZHAO W,et al.Image stitching method by multi-feature constrained alignment and colour adjustment[J].IET Image Processing,2021,15(7):1499-1507.
[16]LIAO T,CHEN J,XU Y.Quality evaluation-based iterativeseam estimation for image stitching[J].Signal,Image and Video Processing,2019,13:1199-1206.
[17]ZHANG J,GAO Y,XU Y,et al.A simple yet effective image stitching with computational suture zone[J].The Visual Computer,2023,39(10):4915-4928.
[18]CHEN X,YU M,SONG Y.Optimized seam-driven image sti-tching method based on scene depth information[J].Electronics,2022,11(12):1876.
[19]ZHANG J,WANG C,LIU S,et al.Content-aware unsupervised deep homography estimation[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part I 16.Springer International Publishing,2020:653-669.
[20]SHEN X,DARMON F,EFROS A A,et al.Ransac-flow:generic two-stage image alignment[C]//Computer Vision-ECCV 2020:16th European Conference,Glasgow,UK,August 23-28,2020,Proceedings,Part IV 16.Springer International Publishing,2020:618-637.
[21]LI A,GUO J,GUO Y.Image stitching based on semantic planar region consensus[J].IEEE Transactions on Image Processing,2021,30:5545-5558.
[22]NIE L,LIN C,LIAO K,et al.Deep rectangling for image stitching:a learning baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:5740-5748.
[23]XU X Y,YUAN S S,WANG J,et al.Image stitching method based on global and local features [J].Journal of Beijing Institute of Technology,2022,42(5):502-510.
[24]ALCANTARILLA P F,BARTOLI A,DAVISON A J.KAZE features[C]//Computer Vision-ECCV 2012:12th European Conference on Computer Vision,Florence,Italy,October 7-13,2012,Proceedings,Part VI 12.Springer Berlin Heidelberg,2012:214-227.
[25]CHIN T J,YU J,SUTER D.Accelerated hypothesis generation for multistructure data via preference analysis[J].IEEE Tran-sactions on Pattern Analysis and Machine Intelligence,2011,34(4):625-638.
[26]DING C,MA Z.Multi-camera color correction via hybrid histogram matching[J].IEEE Transactions on Circuits and Systems for Video Technology,2020,31(9):3327-3337.
[27]OTSU N.A threshold selection method from gray-level histograms[J].IEEE Transactions on Systems,Man,and Cyberne-tics,1979,9(1):62-66.
[28]LIAO T,LI N.Single-perspective warps in natural image sti-tching[J].IEEE Transactions on Image Processing,2019,29:724-735.
[29]JIA Q,LI Z J,FAN X,et al.Leveraging line-point consistence to preserve structures for wide parallax image stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2021:12186-12195.
[30]LI N,LIAO T,WANG C.Perception-based seam cutting forimage stitching[J].Signal,Image and Video Processing,2018,12:967-974.
[31]VON GIOI R G,JAKUBOWICZ J,MOREL J M,et al.LSD:A line segment detector[J].Image Processing On Line,2012,2:35-55.
[32]NIE L,LIN C,LIAO K,et al.Parallax-Tolerant UnsupervisedDeep Image Stitching[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:7399-7408.
[1] MIN Lihua, DING Tianzhong, JIN Zhengmeng. Deformable Image Registration Model Based on Weighted Bounded Deformation Function [J]. Computer Science, 2024, 51(6): 206-214.
[2] HE Weilong, SU Lingli, GUO Bingxuan, LI Maosen, HAO Yan. Research and Implementation of Dynamic Scene 3D Perception Technology Based on BinocularEstimation [J]. Computer Science, 2024, 51(11A): 240300045-8.
[3] CHANG Zi-ting, SHI Yu-qing, WANG Jun, YU Ming-he, YAO Lan, ZHAO Zhi-bin. Vehicle Speed Measurement Method Based on Binocular Vision [J]. Computer Science, 2021, 48(9): 135-139.
[4] HU Yu-cheng, RUI Ting, YANG Cheng-song, WANG Dong, LIU Xun. Study on Aerial Image Fast Registration from UAV [J]. Computer Science, 2021, 48(8): 134-138.
[5] WEI Dong, LIU Hao, CHEN Gen-long, GONG Xiao-hui. Underwater Image Enhancement Based on Color Correction and Deblurring [J]. Computer Science, 2021, 48(4): 144-150.
[6] TANG Jia-lin, ZHANG Chong, GUO Yan-feng, SU Bing-hua and SU Qing-lang. Color Difference Correction Algorithm Based on Multi Colors Space Information [J]. Computer Science, 2020, 47(6A): 157-160.
[7] PANG Rong,LAI Lin-jing,ZHANG Lei. Grid-driven Bi-directional Image Stitching Algorithm [J]. Computer Science, 2020, 47(3): 130-136.
[8] CHEN Ying, ZHAO Lai-wang, ZHAN Hong-chen, DING Yao. Study on Reconstruction of Indoor 3D Scene Based on Binocular Vision [J]. Computer Science, 2020, 47(11A): 175-177.
[9] ZHAO Zi-yang, JIANG Mu-rong, HUANG Ya-qun, HAO Jian-yu, ZENG Ke. Single Image Depth Estimation Algorithm Based on SFS and Binocular Model [J]. Computer Science, 2019, 46(6A): 161-164.
[10] LI Yin-guo, ZHOU Zhong-kui, BAI Ling. Large-scale Automatic Driving Scene Reconstruction Based on Binocular Image [J]. Computer Science, 2019, 46(11A): 251-254.
[11] LOU Hao-feng, ZHANG Duan. Gaussian Process Assisted CMA-ES Application in Medical Image Registration [J]. Computer Science, 2018, 45(11A): 234-237.
[12] LI Dan, XIAO Li-qing, TIAN Jun and SUN Jin-ping. Multi-images Mosaic Algorithm Based on Improved Phase Correlation and Feature Point Registration [J]. Computer Science, 2018, 45(1): 313-319.
[13] ZHOU Shi-hao and ZHANG Yun. Log-polar Feature Guided Iterative Closest Point Algorithm [J]. Computer Science, 2018, 45(1): 297-306.
[14] ZHENG Wei, JIANG Chen-jiao, LIU Shuai-qi and ZHAO Jie. Improved Chicken Swarm Optimization Algorithm and Its Application in DTI-FA Image Registration [J]. Computer Science, 2018, 45(1): 285-291.
[15] LU Jia-ming and ZHU Zhe. Real-time 4K Panoramic Video Stitching Based on GPU Acceleration [J]. Computer Science, 2017, 44(8): 18-21.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!