Computer Science ›› 2022, Vol. 49 ›› Issue (10): 258-264.doi: 10.11896/jsjkx.211000172

• Artificial Intelligence • Previous Articles     Next Articles

Trajectory Prediction Method Based on Fusion of Graph Interaction and Scene Perception

FANG Yang1, ZHAO Ting2, LIU Qi-lie2, HE Dong3, SUN Kai-wei1, CHEN Qian-bin2   

  1. 1 School of Computer Science and Technology,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    2 School of Communication and Information Engineering,Chongqing University of Posts and Telecommunications,Chongqing 400065,China
    3 School of Electrical Engineering,KAIST,Daejeon 34141,South Korea
  • Received:2021-10-25 Revised:2022-02-28 Online:2022-10-15 Published:2022-10-13
  • About author:FANG Yang,born in 1991,Ph.D,lectu-rer,is a member of China Computer Fe-deration.His main research interests include computer vision and pattern recognition,visual object tracking,lidar-based 3D sensing and perception for AD system.
    ZHAO Ting,born in 1995,postgra-duate.Her main research interests include big data,lidar sensing and trajectory prediction.
  • Supported by:
    Science and Technology Research Program of Chongqing Municipal Education Commission(KJQN202100634),Chongqing Science and Technology Innovation Leading Talent Support Program(CSTCCXLJRC201908),Basic and Advanced Research Projects of CSTC(cstc2019jcyj-zdxmX0008),Science and Technology Research Program of Chongqing Municipal Education Commission(KJZD-K201900605),Young Scientists Fund of the National Natural Science Foundation of China(61806033), Natural Science Foundation of Chongqing, China(cstc2019jcyj-msxmX0021) and Scientific and Technological Innovation Projects of the Construction of the Two Cities Economic Circle in Chengdu Chongqing Region(KJCXZD2020027).

Abstract: To accurately perceive the environment and predict the trajectory of the surrounding traffic participants for autonomous driving,we propose a real-time end-to-end trajectory prediction framework based on bird eye view(BEV) to learn both interaction and scene information simultaneously.The framework consists of two essential modules:graph interaction network and pyramid perception network.The former encodes the interaction patterns among traffic participants through a spatiotemporal graph convolutional network,and the latter adopts a spatiotemporal pyramid network to model the surrounding information and obtain the scene features.Next,interactive features and scene features are fused at a unified scale to perform classification and trajectory prediction tasks.Experiments and analysis on Nuscenes,a large open-source dataset,indicate that the proposed framework achieves a higher classification accuracy of 3.1% and 1.43% less predicted trajectory loss than MotionNet.Hence,our framework outperforms state-of-the-art algorithms in terms of generalization and robustness,and is more in line with perception requirements in actual autonomous driving scenes.

Key words: Trajectory prediction, Spatiotemporal graph convolutional, Spatiotemporal pyramid, Graph interaction encoding, Feature fusion

CLC Number: 

  • TP183
[1]MOZAFFARI S,AL-JARRAH O Y,DIANATI M,et al.DeepLearning-based Vehicle Behaviour Prediction for Autonomous Driving Applications:A Review[J].arXiv:1912.11676,2019.
[2]LEE N,CHOI W,VERNAZA P,et al.DESIRE:Distant Future Prediction in Dynamic Scenes with Interacting Agents[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,IEEE,2017:2165-2174.
[3]ZENG W L,CHEN Y H,YAO R Y,et al.Application of Spatial-Temporal Graph Attention Networks in Trajectory Prediction for Vehicles at Intersections[J].Computer Science,2021,48(S1):334-341.
[4]LI L H,ZHOU B,LIAN J,et al.Research on pedestrian trajectory prediction method based on social attention mechanism[J].Journal on Communications,2020,41(12):175-183.
[5]JUSTS D J,NOVICKIS R,OZOLS K,et al.Bird's-eye viewimage acquisition from simulated scenes using geometric inverse perspective mapping[C]//2020 17th Biennial Baltic Electronics Conference(BEC).Tallinn,2020:1-6.
[6]CHEN S,LIU B,FENG C,et al.3D Point Cloud Processing andLearning for Autonomous Driving[J].IEEE Signal Processing Magazine,2021,38(1):68-86.
[7]LI B L,YANG D,WANG L,et al.Weak Echo Signal Processing of 1 550 nm Coherent Laser Wind Radar[J].Piezoelectrics and Acoustooptics,2022,44(2):333-338.
[8]LEFÉVRE S,VASQUEZ D,LAUGIER C.A survey on motion prediction and risk assessment for intelligent vehicles[J].Robomech Journal,2014,1(1):1-14.
[9]YOU L,HAN X W,HE Z W,et al.Improved Sequence-to-Sequence Model for Short-term Vessel Trajectory Prediction Using AIS Data Streams[J].Computer Science,2020,47(9):169-174.
[10]ZHOU Y,TUZEL O.VoxelNet:End-to-End Learning for Point Cloud Based 3D Object Detection[C]//2018 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR).Salt Lake City,IEEE,2018:4490-4499.
[11]LUO W,YANG B,URTASUN R.Fast and Furious:Real Time End-to-End 3D Detection,Tracking and Motion Forecasting with a Single Convolutional Net[C]//2018 IEEE/CVF Confe-rence on Computer Vision and Pattern Recognition(CVPR).Salt Lake City,IEEE,2018:3569-3577.
[12]LEFEVRE S,VASQUEZ D,LAUGIER C.A survey on motion prediction and risk assessment for intelligent vehicles[J].ROBOMECH Journal,2014,1(1):1-9.
[13]SHI S,WANG X,LI H.PointRCNN:3D Object Proposal Ge-neration and Detection from Point Cloud[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:770-779.
[14]ZENG W Y,WANG S L,LIAO R J,et al.DSDNet:Deep Structured self-Driving Network[C]//2020 European Conference Computer Vision(ECCV).Glasgow,2020:156-172.
[15]SCHREIBER M,HOERMANN S,DIETMAYER K.Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks[C]//2019 International Conference on Robotics and Automation(ICRA).Montreal,2019:9299-9305.
[16]CHEN X,MA H,WAN J,et al.Multi-View 3D Object Detection Network for Autonomous Driving[C]//2017 IEEE Confe-rence on Computer Vision and Pattern Recognition(CVPR).IEEE,2017:6526-6534.
[17]YUAN Z,SONG X,BAI L,et al.Temporal-Channel Transfor-mer for 3D Lidar-Based Video Object Detection for Autonomous Driving[J].IEEE Transactions on Circuits and Systems for Video Technology,2022,32(4):2068-2078.
[18]CUI H,RADOSAVLJEVIC V,CHOU F C,et al.MultimodalTrajectory Predictions for Autonomous Driving using Deep Convolutional Networks[C]//2019 International Conference on Robotics and Automation(ICRA).2019:2090-2096.
[19]XU J,XIAO L,ZHAO D,et al.Trajectory Prediction for Auto-nomous Driving with Topometric Map[J].arXiv:2105.03869,2021.
[20]ZENG W Y,LUO W J,SUO S,et al.End-to-end Interpretable Neural Motion Planner[C]//2019 IEEE Conference on Compu-ter Vision and Pattern Recognition(CVPR).IEEE,2019:8660-8669.
[21]CASAS S,LUO W J,URTASUN R.IntentNet:Learning toPredict Intention from Raw Sensor Data[C]//CoRL 2018.2018:947-956.
[22]ZHANG Z S,GAO J Y,MAO J H,et al.STINet:Spatio-Temporal-Interactive Network for Pedestrian Detection and Trajectory Prediction[C]//2020 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2020:11346-11355.
[23]TRAN D,BOURDEV L,FERGUS R,et al.Learning Spatiotemporal Features with 3D Convolutional Networks[C]//IEEE International Conference on Computer Vision.IEEE,2015:4489-4497.
[24]WU Z,PAN S,CHEN F,et al.A Comprehensive Survey onGraph Neural Networks[J].IEEE Transactions on Neural Networks and Learning Systems,2019,32(1):4-24.
[25]MARINO K,SALAKHUTDINOV R,GUPTA A.The MoreYou Know:Using Knowledge Graphs for Image Classification[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu,IEEE,2017:20-28.
[26]SHEN Y,LI H,YI S,et al.Person Re-identification with Deep Similarity-Guided Graph Neural Network[C]//European Conference on Computer Vision.Cham:Springer,2018:508-526.
[27]WU P,CHEN S,METAXAS D.MotionNet:Joint Perceptionand Motion Prediction for Autonomous Driving Based on Bird's Eye View Maps[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle,IEEE,2020:11382-11392.
[28]LIU X,QI C R,GUIBAS L J.FlowNet3D:Learning Scene Flow in 3D Point Clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:529-537.
[29]GU X,WANG Y,WU C,et al.HPLFlowNet:Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach,IEEE,2019:3254-3263.
[30]SCHREIBER M,HOERMANN S,DIETMAYER K.Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks[C]//2019 International Conference on Robotics and Automation(ICRA).Montreal,2019:9299-9305.
[31]SCHÖLKOPF B,TSUDA K,VERT J.A Primer on KernelMethods[M].Massachusetts:MIT Press,2004.
[1] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[2] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[3] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[4] YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[5] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[6] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[7] LAN Ling-xiang, CHI Ming-min. Remote Sensing Change Detection Based on Feature Fusion and Attention Network [J]. Computer Science, 2022, 49(6): 193-198.
[8] LI Fa-guang, YILIHAMU·Yaermaimaiti. Real-time Detection Model of Insulator Defect Based on Improved CenterNet [J]. Computer Science, 2022, 49(5): 84-91.
[9] DONG Qi-da, WANG Zhe, WU Song-yang. Feature Fusion Framework Combining Attention Mechanism and Geometric Information [J]. Computer Science, 2022, 49(5): 129-134.
[10] LI Peng-zu, LI Yao, Ibegbu Nnamdi JULIAN, SUN Chao, GUO Hao, CHEN Jun-jie. Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion [J]. Computer Science, 2022, 49(5): 206-211.
[11] FAN Xin-nan, ZHAO Zhong-xin, YAN Wei, YAN Xi-jun, SHI Peng-fei. Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism [J]. Computer Science, 2022, 49(5): 50-57.
[12] GAO Xin-yue, TIAN Han-min. Droplet Segmentation Method Based on Improved U-Net Network [J]. Computer Science, 2022, 49(4): 227-232.
[13] XU Tao, CHEN Yi-ren, LYU Zong-lei. Study on Reflective Vest Detection for Apron Workers Based on Improved YOLOv3 Algorithm [J]. Computer Science, 2022, 49(4): 239-246.
[14] XU Hua-jie, QIN Yuan-zhuo, YANG Yang. Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module [J]. Computer Science, 2022, 49(4): 209-214.
[15] YANG Xiao-yu, YIN Kang-ning, HOU Shao-qi, DU Wen-yi, YIN Guang-qiang. Person Re-identification Based on Feature Location and Fusion [J]. Computer Science, 2022, 49(3): 170-178.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!