计算机科学 ›› 2022, Vol. 49 ›› Issue (6A): 441-446.doi: 10.11896/jsjkx.210600043

• 图像处理&多媒体技术 • 上一篇    下一篇

基于CUDA核函数的多路视频图像拼接加速算法

刘云1, 董守杰2   

  1. 1 南京莱斯信息技术股份有限公司 南京 210001
    2 中国民用航空华北地区空中交通管理局 北京 100710
  • 出版日期:2022-06-10 发布日期:2022-06-08
  • 通讯作者: 刘云(10936426@qq.com)

Acceleration Algorithm of Multi-channel Video Image Stitching Based on CUDA Kernel Function

LIU Yun1, DONG Shou-jie2   

  1. 1 Nanjing Les Information Technology Co.,Ltd,Nanjing 210001,China
    2 North China Air Traffic Administration of Civil Aviation of China,Beijing 100710,China
  • Online:2022-06-10 Published:2022-06-08
  • About author:LIU Yun,born in 1991,postgraduate,intermediate engineer.Her main inte-rests include video image processing and moving target recognition algorithm.

摘要: 目前,通用机场数量飞速增长,远程塔台发展迅猛,多路视频实时拼接技术不断改进。视频拼接算法大部分是改进特征点提取算法、特征点匹配算法和图像融合算法,改进特征点提取算法使得提取特征点更准确,改进图像融合算法使得全景图上各路图像色彩一致并且消除拼接缝,最后使用图形处理器实现加速。基于图像拼接整体计算流程,从数学角度计算原始图像到全景图像像素坐标变换矩阵和像素值变换矩阵。视频拼接时,基于像素坐标变换矩阵和像素值变换矩阵,使用核函数计算原始图像的像素点经过矩阵变换后的坐标和像素值,设置核函数被调用时的线程配置,充分利用图形处理器的并行计算能力,实现图像拼接加速。实验结果表明,拼接8张1080*1920的图像耗时约22 ms。视频拼接的核心技术包括拉流、解码、拼接、编码传输,文章结尾对其中的拉流和解码技术给出了建议。

关键词: 核函数, 矩阵变换, 视频拼接, 统一设备架构, 图形处理器

Abstract: With the rapid growth of the number of general airports and the rapid development of remote towers,multi-channel real-time video stitching technology is constantly improved.Most of the video mosaic algorithms are based on the improvement of feature point extraction algorithm,feature point matching algorithm and image fusion algorithm.The improved feature point extraction algorithm makes the feature points more accurate.The improved image fusion algorithm makes the color of each image on the panorama consistent and eliminates the stitching seam.Finally,the GPU is used to accelerate the process.This paper focuses on the overall calculation process of image mosaic,from the mathematical point of view to calculate the pixel coordinate transformation matrix and pixel value transformation matrix from the original image to panoramic image.In video stitching,based on the pixel coordinate transformation matrix of pixel value transformation matrix,the kernel function is used to calculate the coordinates and pixel values after matrix transformation.Set the thread configuration when the kernel function is called.The parallel computing ability of the graphics processor is fully utilized to accelerate the image stitching.The experimental results show that the stitching time of 8-channel 1080 * 1920 videos is about 22 ms.The core technology of video stitching includes streaming,decoding,stitching,coding and transmission.At the end of the paper,some suggestions on streaming and decoding technology are put forward.

Key words: Compute unified device architecture, Graphic Processing Unit, Kernel function, Matrix transformation, Video stitching

中图分类号: 

  • TP399
[1] Civil Aviation Administration of China.Strategic Conception of Building a Powerful Civil Aviation Country[Z].Baijing,2010.
[2] HE H,HE X.Analysis on the Influence of General Airport on County Economic Development[J].Science Technology and Industry,2020,20(12):154-159.
[3] KRAISS K,KUHLEN T.Virtual reality:principles and applications.From sensors to situstion awareness[C]//DLR-Mitteilung.Koln DLR,1996:96-112.
[4] CHEN D Q,LI T T,GUO X,et al.Improved SIFT neighborhood Voting image matching algorithm[J].Computer Enginee-ring and Design,2020,41(1):162-168.
[5] GUO X,YANG J,LIN H.Image registration method based on improved SIFT algorithm and essential matrix estimation[C]//IEEE International Conference on Information & Automation.Macau:IEEE,2017:814-819.
[6] HYUN B J,JAE E J.Feature-point extraction based on an in-proved SIFT algorithm[C]//International Conference on Control,Automation an Systems.Jeju:IEEE,2017:345-350.
[7] ZHOU Z W,YUAN F W,ZHANG K,et al.Image Matching Method Combining SURF and FLANN Algorithm[J].Intelligent Computer and Applications,2019,9(6):160-163,167.
[8] TANG Y F,WANG Z J,ZHANG Z X.Registration of SandDune Images Using an Improved SIFT and SURF Algorithm[J].Journal of Tsinghua University(Science and Technology),2021,61(2):161-169.
[9] DI G H,SU F L,YANG H X,et al.ISAR image scattering center association based on speeded-up robust features[J].Multimedia Tools and Applications,2020,79(9):5065-5082.
[10] ZHANG L T,HUANG X L,LU L L,et al.An Improved Harris Corner Detection Algorithm Based on Similar Pixels[J].Science of Surveying and Mapping,2019,44(7):111-115.
[11] WANG L,ZHOU X Z,YAN H.Real-time SIFT AlgorithmBased on GPU[J].Computer Science,2020,47(8):105-111.
[12] YANG Z F,YAN L.Technologies of Image Mosaic Based onImproved SURF Algorithm[J].Journal of Wuhan Institute of Technology,2021,43(2):223-226,231.
[13] MACHADO L,MATTHEW T,JIE L,et al.Nonrigid registration of 3D ultrasound for neurosurgery using automatic feature detection and matching[J].International Journal of Computer Assisted Radiology and Surgery,2018,13(10):1525-1538.
[14] KOU Q Q,CHENG D Q,ZHUANG H D,et al.Cross-complementary local binary pattern for robust texture classication[J].IEEE Signal Processing Letters,2019,26(1):129-133.
[15] LI Y F,LI G Z,GU S H,et al.Image mosaic algorithm based on area blocking and SIFT[J].Optics and Precision Engineering,2016,24(5):1197-1205.
[16] CHEN X.An Algorithm Development Environment for Problem-Solving[C]//International Conference on Computational Problem-Solving.IEEE,2010:85-90.
[17] WANG J L,ZHOU Z F.Image Matching Algorithm Based on SIFT Feature Extraction and FLANN Search[J].Computer Measurement and Control,2018,26(2):175-178.
[18] SHAO J D,YANG S,CHEN L.UAV Image Matching Algorithm Based on Improved SIFT Algorithm and Two-stage Feature Matching[J].Computer Science,2019,46(6):316-321.
[19] ZHANG M Y,WANG Y Y,HOU X H,et al.Image Stitching Algorithm Based on ORB and Improved RANSAC[J].Computer Science,2019,46(S2):294-298.
[20] LI X G,REN C,ZHANG T X,et al.Unmanned Aerial Vehicle Image Matching Based on Improved RANSAC Algorithm and SURF Algorithm[J].The International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2020:67-70.
[21] QU Z,QIAO G Y,LIN S P.A Fast Image Stitching Algorithm Eliminates Seam Line and Ghosting[J].Computer Science,2015,42(3):280-283.
[22] QU Z,LI X L.Algorithm of Eliminating Image Stitching LineBased on Improved IGG Model[J].Computer Science,2017,44(12):274-278.
[23] CHENG Y F.The Algorithm of Fast Image Mosaic Based on Stitching Seam Elimination and Panorama Alignment[D].Chongqing:Chongqing University of Posts and Telecommunications,2020.
[24] XU S Y.Research on Image Processing Algorithms for 4K Ultra HD Endoscopes Based on CUDA[D].Zhejiang:Zhejiang University,2020.
[25] WANG Z,ZHANG B K,WANG J,et al.CUDA-Based Seamless Stitching Method for Multi-Camera Real-Time HD Video[J].Computer Applications and Software,2016,33(2):123-128.
[26] LI X.Research On Lane Detection Algorithm Based On CUDA Acceleration[D].Dalian:Dalian University of Technology,2020.
[27] WANG Y K,LIU R,WEN T X,et al.GPU Acceleration for Kernel Regression Algorithm of Three-Dimensional Medical Image[J].Chinese Journal of Medical Physics,2018,35(12):1417-1425.
[28] YU M H,WANG S T,LI Y C,et al.Distortion AlgorithmOpenCL Parallel Acceleration[J].Remote Sensing Information,2019,34(3):88-92.
[1] 杨思燕,贺国旗,刘如意.
基于SIFT算法的大场景视频拼接算法及优化
Video Stitching Algorithm Based on SIFT and Its Optimization
计算机科学, 2019, 46(7): 286-291. https://doi.org/10.11896/j.issn.1002-137X.2019.07.044
[2] 张善文, 文国秋, 张乐园, 李佳烨.
基于核函数的稀疏属性选择算法
Sparse Feature Selection Algorithm Based on Kernel Function
计算机科学, 2019, 46(2): 62-67. https://doi.org/10.11896/j.issn.1002-137X.2019.02.010
[3] 朱超, 吴素萍.
并行Harris特征点检测算法
Parallel Harris Feature Point Detection Algorithm
计算机科学, 2019, 46(11A): 289-293.
[4] 陈俊芬, 张明, 何强.
基于启发式确定类数的NJW谱聚类算法
Heuristically Determining Cluster Numbers Based NJW Spectral Clustering Algorithm
计算机科学, 2018, 45(11A): 474-479.
[5] 陈鹏, 张建伟.
结合核函数与非线性偏微分方程的图像去噪方法
Image Denoising Method Combining Kernel Function and Nonlinear Partial Differential Equation
计算机科学, 2018, 45(11): 278-282. https://doi.org/10.11896/j.issn.1002-137X.2018.11.044
[6] 叶秋萍,张红英.
基于一种新的核函数的模糊粗糙集
Fuzzy Rough Sets Based on New Kernel Functions
计算机科学, 2017, 44(9): 70-73. https://doi.org/10.11896/j.issn.1002-137X.2017.09.014
[7] 唐滔,彭林,黄春,杨灿群.
面向存储层次设计优化的GPU程序性能分析
Performance Analysis of GPU Programs Towards Better Memory Hierarchy Design
计算机科学, 2017, 44(12): 1-10. https://doi.org/10.11896/j.issn.1002-137X.2017.12.001
[8] 孟志青,马珂,郑英.
基于核函数技术的时尚服装需求预测方法
Forecasting Method for Fashion Clothing Demand Based on Kernel Functions Technology
计算机科学, 2016, 43(Z11): 455-460. https://doi.org/10.11896/j.issn.1002-137X.2016.11A.102
[9] 张静,熊承义,高志荣.
基于GPU的压缩感知重构算法的设计与实现
Implementation for Compressed Sensing Reconstruction Algorithm Based on GPU
计算机科学, 2016, 43(8): 318-322. https://doi.org/10.11896/j.issn.1002-137X.2016.08.065
[10] 甘玲,邹宽中,刘肖.
基于PCA降维的多特征级联的行人检测
Pedestrian Detection Based on PCA Dimension Reduction of Multi-feature Cascade
计算机科学, 2016, 43(6): 308-311. https://doi.org/10.11896/j.issn.1002-137X.2016.06.061
[11] 梁宏涛,徐建良,许可.
组合核相关向量机的软件可靠性预测模型
Software Reliability Prediction Model Based on Hybrid Kernels RVM
计算机科学, 2016, 43(11): 257-259. https://doi.org/10.11896/j.issn.1002-137X.2016.11.050
[12] 许洁,梁久祯,吴秦,李敏.
核典型相关分析特征融合方法及应用
Kernel Canonical Correlation Analysis Feature Fusion Method and Application
计算机科学, 2016, 43(1): 35-39. https://doi.org/10.11896/j.issn.1002-137X.2016.01.008
[13] 韩磊,徐 波,黄向生,张彦峰.
基于GPU的散斑三维重建系统
Speckle Projection Systems Based on GPU
计算机科学, 2015, 42(8): 294-299.
[14] 田浩兵,朱嘉钢,陆 晓.
基于特征贡献度加权高斯核函数的粗糙one-class支持向量机
WFCD-based Rough Set One-class Support Vector Machine
计算机科学, 2015, 42(6): 239-242. https://doi.org/10.11896/j.issn.1002-137X.2015.06.050
[15] 王裴岩,蔡东风.
基于统计检验的核函数度量方法研究
Statistical Testing Based Research on Kernel Evaluation Measures
计算机科学, 2015, 42(4): 199-205. https://doi.org/10.11896/j.issn.1002-137X.2015.04.040
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!