计算机科学 ›› 2022, Vol. 49 ›› Issue (10): 258-264.doi: 10.11896/jsjkx.211000172

• 人工智能 • 上一篇    下一篇

基于图交互与场景感知融合的轨迹预测方法

方阳1, 赵婷2, 刘期烈2, 贺侗3, 孙开伟1, 陈前斌2   

  1. 1 重庆邮电大学计算机科学与技术学院 重庆 400065
    2 重庆邮电大学通信与信息工程学院 重庆 400065
    3 韩国科学技术院(KAIST)电气工程学院 大田 34141
  • 收稿日期:2021-10-25 修回日期:2022-02-28 出版日期:2022-10-15 发布日期:2022-10-13
  • 通讯作者: 赵婷(S190101071@stu.cqupt.edu.cn)
  • 作者简介:(fangyang@cqupt.edu.cn)
  • 基金资助:
    重庆市教委青年项目(KJQN202100634);重庆市科技创新领军人才支持计划(CSTCCXLJRC201908);重庆市自然基金重点项目(cstc2019jcyj-zdxm0008);重庆市教委重点项目(KJZD-K201900605);国家自然科学基金青年科学基金项目(61806033);重庆市自然科学基金面上项目(cstc2019jcyj-msxmX0021);“成渝地区双城经济圈建设”科技创新项目(KJCXZD2020027)

Trajectory Prediction Method Based on Fusion of Graph Interaction and Scene Perception

FANG Yang1, ZHAO Ting2, LIU Qi-lie2, HE Dong3, SUN Kai-wei1, CHEN Qian-bin2   

  1. 1 School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    3 School of Electrical Engineering,KAIST,Daejeon 34141,South Korea
  • Received:2021-10-25 Revised:2022-02-28 Online:2022-10-15 Published:2022-10-13
  • About author:FANG Yang,born in 1991,Ph.D,lectu-rer,is a member of China Computer Fe-deration.His main research interests include computer vision and pattern recognition,visual object tracking,lidar-based 3D sensing and perception for AD system.
    ZHAO Ting,born in 1995,postgra-duate.Her main research interests include big data,lidar sensing and trajectory prediction.
  • Supported by:
    Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN202100634),Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908),Basic and Advanced Research Projects of CSTC(cstc2019jcyj-zdxmX0008),Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-K201900605),Young Scientists Fund of the National Natural Science Foundation of China(61806033), Natural Science Foundation of Chongqing, China(cstc2019jcyj-msxmX0021) and Scientific and Technological Innovation Projects of the Construction of the Two Cities Economic Circle in Chengdu Chongqing Region(KJCXZD2020027).

摘要: 在自动驾驶中,精确的环境感知和对周围交通参与者的轨迹预测对道路安全至关重要。基于此,提出了基于鸟瞰图(Bird Eye View,BEV)的实时端到端轨迹预测框架来同时学习交互和场景信息。该框架主要由图交互网络和金字塔感知网络两个模块组成,前者通过时空图卷积网络对交通参与者之间的交互模式进行编码,后者采用时空金字塔网络对周围信息进行场景建模以获取场景特征。然后,对交互特征和场景特征进行单一尺度融合,从而进行分类和轨迹预测任务。在大规模开源数据集NuScenes上的实验和分析表明,与当前先进算法(MotionNet)相比,所提框架平均类别准确度提高了3.1%,轨迹预测平均误差在行驶速度>5m/s时降低了1.43%。此实验结果表明,所提模型具有更好的泛化性和鲁棒性,更符合实际自动驾驶环境中的轨迹预测需求。

关键词: 轨迹预测, 时空图卷积, 时空金字塔, 图交互编码, 特征融合

Abstract: To accurately perceive the environment and predict the trajectory of the surrounding traffic participants for autonomous driving,we propose a real-time end-to-end trajectory prediction framework based on bird eye view(BEV) to learn both interaction and scene information simultaneously.The framework consists of two essential modules:graph interaction network and pyramid perception network.The former encodes the interaction patterns among traffic participants through a spatiotemporal graph convolutional network,and the latter adopts a spatiotemporal pyramid network to model the surrounding information and obtain the scene features.Next,interactive features and scene features are fused at a unified scale to perform classification and trajectory prediction tasks.Experiments and analysis on Nuscenes,a large open-source dataset,indicate that the proposed framework achieves a higher classification accuracy of 3.1% and 1.43% less predicted trajectory loss than MotionNet.Hence,our framework outperforms state-of-the-art algorithms in terms of generalization and robustness,and is more in line with perception requirements in actual autonomous driving scenes.

Key words: Trajectory prediction, Spatiotemporal graph convolutional, Spatiotemporal pyramid, Graph interaction encoding, Feature fusion

中图分类号: 

  • TP183
[1]MOZAFFARI S,AL-JARRAH O Y,DIANATI M,et al.DeepLearning-based Vehicle Behaviour Prediction for Autonomous Driving Applications:A Review[J].arXiv:1912.11676,2019.
[2]LEE N,CHOI W,VERNAZA P,et al.DESIRE:Distant Future Prediction in Dynamic Scenes with Interacting Agents[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,IEEE,2017:2165-2174.
[3]ZENG W L,CHEN Y H,YAO R Y,et al.Application of Spatial-Temporal Graph Attention Networks in Trajectory Prediction for Vehicles at Intersections[J].Computer Science,2021,48(S1):334-341.
[4]LI L H,ZHOU B,LIAN J,et al.Research on pedestrian trajectory prediction method based on social attention mechanism[J].Journal on Communications,2020,41(12):175-183.
[5]JUSTS D J,NOVICKIS R,OZOLS K,et al.Bird's-eye viewimage acquisition from simulated scenes using geometric inverse perspective mapping[C]//2020 17th Biennial Baltic Electronics Conference(BEC).Tallinn,2020:1-6.
[6]CHEN S,LIU B,FENG C,et al.3D Point Cloud Processing andLearning for Autonomous Driving[J].IEEE Signal Processing Magazine,2021,38(1):68-86.
[7]LI B L,YANG D,WANG L,et al.Weak Echo Signal Processing of 1 550 nm Coherent Laser Wind Radar[J].Piezoelectrics and Acoustooptics,2022,44(2):333-338.
[8]LEFÉVRE S,VASQUEZ D,LAUGIER C.A survey on motion prediction and risk assessment for intelligent vehicles[J].Robomech Journal,2014,1(1):1-14.
[9]YOU L,HAN X W,HE Z W,et al.Improved Sequence-to-Sequence Model for Short-term Vessel Trajectory Prediction Using AIS Data Streams[J].Computer Science,2020,47(9):169-174.
[10]ZHOU Y,TUZEL O.VoxelNet:End-to-End Learning for Point Cloud Based 3D Object Detection[C]//2018 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR).Salt Lake City,IEEE,2018:4490-4499.
[11]LUO W,YANG B,URTASUN R.Fast and Furious:Real Time End-to-End 3D Detection,Tracking and Motion Forecasting with a Single Convolutional Net[C]//2018 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR).Salt Lake City,IEEE,2018:3569-3577.
[12]LEFEVRE S,VASQUEZ D,LAUGIER C.A survey on motion prediction and risk assessment for intelligent vehicles[J].ROBOMECH Journal,2014,1(1):1-9.
[13]SHI S,WANG X,LI H.PointRCNN:3D Object Proposal Ge-neration and Detection from Point Cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:770-779.
[14]ZENG W Y,WANG S L,LIAO R J,et al.DSDNet:Deep Structured self-Driving Network[C]//2020 European Conference Computer Vision(ECCV).Glasgow,2020:156-172.
[15]SCHREIBER M,HOERMANN S,DIETMAYER K.Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks[C]//2019 International Conference on Robotics and Automation(ICRA).Montreal,2019:9299-9305.
[16]CHEN X,MA H,WAN J,et al.Multi-View 3D Object Detection Network for Autonomous Driving[C]//2017 IEEE Confe-rence on Computer Vision and Pattern Recognition(CVPR).IEEE,2017:6526-6534.
[17]YUAN Z,SONG X,BAI L,et al.Temporal-Channel Transfor-mer for 3D Lidar-Based Video Object Detection for Autonomous Driving[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(4):2068-2078.
[18]CUI H,RADOSAVLJEVIC V,CHOU F C,et al.MultimodalTrajectory Predictions for Autonomous Driving using Deep Convolutional Networks[C]//2019 International Conference on Robotics and Automation(ICRA).2019:2090-2096.
[19]XU J,XIAO L,ZHAO D,et al.Trajectory Prediction for Auto-nomous Driving with Topometric Map[J].arXiv:2105.03869,2021.
[20]ZENG W Y,LUO W J,SUO S,et al.End-to-end Interpretable Neural Motion Planner[C]//2019 IEEE Conference on Compu-ter Vision and Pattern Recognition(CVPR).IEEE,2019:8660-8669.
[21]CASAS S,LUO W J,URTASUN R.IntentNet:Learning toPredict Intention from Raw Sensor Data[C]//CoRL 2018.2018:947-956.
[22]ZHANG Z S,GAO J Y,MAO J H,et al.STINet:Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction[C]//2020 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020:11346-11355.
[23]TRAN D,BOURDEV L,FERGUS R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//IEEE International Conference on Computer Vision.IEEE,2015:4489-4497.
[24]WU Z,PAN S,CHEN F,et al.A Comprehensive Survey onGraph Neural Networks[J].IEEE Transactions on Neural Networks and Learning Systems,2019,32(1):4-24.
[25]MARINO K,SALAKHUTDINOV R,GUPTA A.The MoreYou Know:Using Knowledge Graphs for Image Classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,IEEE,2017:20-28.
[26]SHEN Y,LI H,YI S,et al.Person Re-identification with Deep Similarity-Guided Graph Neural Network[C]//European Conference on Computer Vision.Cham:Springer,2018:508-526.
[27]WU P,CHEN S,METAXAS D.MotionNet:Joint Perceptionand Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle,IEEE,2020:11382-11392.
[28]LIU X,QI C R,GUIBAS L J.FlowNet3D:Learning Scene Flow in 3D Point Clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:529-537.
[29]GU X,WANG Y,WU C,et al.HPLFlowNet:Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:3254-3263.
[30]SCHREIBER M,HOERMANN S,DIETMAYER K.Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks[C]//2019 International Conference on Robotics and Automation(ICRA).Montreal,2019:9299-9305.
[31]SCHÖLKOPF B,TSUDA K,VERT J.A Primer on KernelMethods[M].Massachusetts:MIT Press,2004.
[1] 张颖涛, 张杰, 张睿, 张文强.
全局信息引导的真实图像风格迁移
Photorealistic Style Transfer Guided by Global Information
计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[2] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[3] 陈永平, 朱建清, 谢懿, 吴含笑, 曾焕强.
基于外接圆半径差损失的实时安全帽检测算法
Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss
计算机科学, 2022, 49(6A): 424-428. https://doi.org/10.11896/jsjkx.220100252
[4] 孙洁琪, 李亚峰, 张文博, 刘鹏辉.
基于离散小波变换的双域特征融合深度卷积神经网络
Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation
计算机科学, 2022, 49(6A): 434-440. https://doi.org/10.11896/jsjkx.210900199
[5] 郁舒昊, 周辉, 叶春杨, 王太正.
SDFA:基于多特征融合的船舶轨迹聚类方法研究
SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion
计算机科学, 2022, 49(6A): 256-260. https://doi.org/10.11896/jsjkx.211100253
[6] 杨玥, 冯涛, 梁虹, 杨扬.
融合交叉注意力机制的图像任意风格迁移
Image Arbitrary Style Transfer via Criss-cross Attention
计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[7] 蓝凌翔, 池明旻.
基于特征注意力融合网络的遥感变化检测研究
Remote Sensing Change Detection Based on Feature Fusion and Attention Network
计算机科学, 2022, 49(6): 193-198. https://doi.org/10.11896/jsjkx.210500058
[8] 邵延华, 李文峰, 张晓强, 楚红雨, 饶云波, 陈璐.
基于时空图卷积和注意力模型的航拍暴力行为识别
Aerial Violence Recognition Based on Spatial-Temporal Graph Convolutional Networks and Attention Model
计算机科学, 2022, 49(6): 254-261. https://doi.org/10.11896/jsjkx.210400272
[9] 范新南, 赵忠鑫, 严炜, 严锡君, 史朋飞.
结合注意力机制的多尺度特征融合图像去雾算法
Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism
计算机科学, 2022, 49(5): 50-57. https://doi.org/10.11896/jsjkx.210400093
[10] 李发光, 伊力哈木·亚尔买买提.
基于改进CenterNet的航拍绝缘子缺陷实时检测模型
Real-time Detection Model of Insulator Defect Based on Improved CenterNet
计算机科学, 2022, 49(5): 84-91. https://doi.org/10.11896/jsjkx.210400142
[11] 董奇达, 王喆, 吴松洋.
结合注意力机制与几何信息的特征融合框架
Feature Fusion Framework Combining Attention Mechanism and Geometric Information
计算机科学, 2022, 49(5): 129-134. https://doi.org/10.11896/jsjkx.210300180
[12] 李鹏祖, 李瑶, Ibegbu Nnamdi JULIAN, 孙超, 郭浩, 陈俊杰.
基于多特征融合的重叠组套索脑功能超网络构建及分类
Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion
计算机科学, 2022, 49(5): 206-211. https://doi.org/10.11896/jsjkx.210300049
[13] 许华杰, 秦远卓, 杨洋.
基于多级特征融合与注意力模块的场景识别方法
Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module
计算机科学, 2022, 49(4): 209-214. https://doi.org/10.11896/jsjkx.210100135
[14] 高心悦, 田汉民.
基于改进U-Net网络的液滴分割方法
Droplet Segmentation Method Based on Improved U-Net Network
计算机科学, 2022, 49(4): 227-232. https://doi.org/10.11896/jsjkx.210300193
[15] 徐涛, 陈奕仁, 吕宗磊.
基于改进YOLOv3的机坪工作人员反光背心检测研究
Study on Reflective Vest Detection for Apron Workers Based on Improved YOLOv3 Algorithm
计算机科学, 2022, 49(4): 239-246. https://doi.org/10.11896/jsjkx.210200119
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!