计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (6A): 250700065-10.doi: 10.11896/jsjkx.250700065

• 图像处理&多媒体技术 • 上一篇    下一篇

基于多模态特征融合的智能识别方法

钟浩1, 孔庆轩2, 蔡先庆1, 黎志忠1, 孙浩1   

  1. 1 四川省交通勘察设计研究院有限公司 成都 610031
    2 四川成绵苍巴高速公路有限责任公司成都分公司 成都 610213
  • 出版日期:2026-06-16 发布日期:2026-06-12
  • 通讯作者: 孔庆轩(404971210@qq.com)
  • 作者简介:(912815346@qq.com)

Intelligent Recognition Method Based on Multimodal Feature Fusion

ZHONG Hao1, KONG Qingxuan2, CAI Xianqing1, LI Zhizhong1, SUN Hao1   

  1. 1 Sichuan Provincial Transportation Survey and Design Research Institute Co.,Ltd.,Chengdu 610031,China
    2 Sichuan ChengmianCangba Expressway Company Limited Chengdu Branch,Chengdu 610213,China
  • Published:2026-06-16 Online:2026-06-12
  • About author:ZHONG Hao,born in 1997,postgra-duate.His main research interests include intelligent recognition and intelligent analysis.
    KONG Qingxuan,born in 1994,postgraduate.His main research interests include intelligent recognition and intelligent analysis.

PDF (PC)

43

摘要： 针对复杂交通场景中光照变化、姿态差异与遮挡导致的车辆识别性能下降问题,提出轻量级多模态融合框架 MM-ASTFL。其核心创新体现在三重注意力机制:一是自适应通道-空间注意力(ACSA),动态加权 MobileNetV2 特征,显著增强车牌、车灯等关键区域表征;二是时间感知注意力增强 LSTM(TA-LSTM),以多头自注意力捕获时序关键帧,精准刻画变道、转弯等驾驶行为;三是跨模态交叉注意力,实现视觉-时序信息的双向引导与深度聚合。在VeRi-776上的实验表明,MM-ASTFL车辆重识别的mAP达89.8%、Rank-1达92.6%,较SOTA提升1.7%与1.6%;驾驶行为分类的F1-score达92.4%,提升2.2%;轨迹预测的ADE与FDE分别降至0.68 m与1.25 m,误差降低13.3%与15.4%。消融实验证实各模块均带来显著增益,可视化分析进一步验证其在逆光、遮挡等极端条件下的鲁棒性,为智能交通系统提供了高效、可靠的解决方案。

关键词: 多模态特征融合, 智能交通系统, 车辆重识别, 驾驶行为分类, 轨迹预测, 自适应注意力机制, 交叉注意力融合

Abstract: To addresses the issue of degraded vehicle recognition performance caused by changes in lighting,pose differences,and occlusions in complex traffic scenarios,this paper proposes a lightweight multi-modal fusion framework,called MM-ASTFL.Its core innovation lies in a triple attention mechanism.First,Adaptive Channel-Space Attention(ACSA),which dynamically weights MobileNetV2 features to significantly enhance the representation of key areas such as license plates and headlights.Second,Time-Aware Attention-Enhanced LSTM(TA-LSTM) uses multi-head self-attention to capture temporal key frames,accurately depicting driving behaviours such as lane changes and turns.Third,cross-modal cross-attention achieves bidirectional guidance and deep aggregation of visual and temporal information.Experiments on VeRi-776 demonstrate that MM-ASTFL achieves a vehicle re-identification mAP of 89.8% and Rank-1 of 92.6%,representing improvements of 1.7% and 1.6% over SOTA;driving behaviour classification F1-score reaches 92.4%,an increase of 2.2%;trajectory prediction ADE and FDE are reduced to 0.68 m and 1.25 m,respectively,with error reductions of 13.3% and 15.4%.Ablation experiments confirm that each module contributes significant gains,and visualisation analysis further validates its robustness under extreme conditions such as backlighting and occlusion,providing an efficient and reliable solution for intelligent transportation systems.

Key words: Multimodal feature fusion, Intelligent transportation systems, Vehicle re-identification, Driving behavior classification, Trajectory prediction, Adaptive attention mechanism, Cross-attention fusion

中图分类号: 

  • TP309
[1] HONG J,MA S,LI K,et al.Vehicle identification and battery voltage prediction using the long short-term memory neural networks for unknown real-world charging pile data oriented to vehicle-pile interaction[J].Journal of Energy Storage,2025,126:116835.
[2] ZHENG Y,WU S,LAI H,et al.A passive radar localization system via accelerated atomic norm optimization and Kolmogorov-Arnold Networks[J].Measurement,2025,253(PA):117441.
[3] ROH J E,IM C,JEONG W,et al.Computation-efficient quantum convolutional neural networks for autonomous driving applications[J].The Journal of Supercomputing,2025,81(8):1033.
[4] VINCENT B.Highway Administration to explore how AI and blockchain can transform transportation[EB/OL].https://www.nextgov.com/emerging-tech/2020/02/highway-administration-explore-how-ai-and-blockchain-can-transform-transportation/162919/.
[5] GAO A,ZHANG G.Research on Vehicle Identification Based on Multi-modal Fusion[J].International Journal on Transport Development and Integration,2024,8(3):371-382.
[6] PAN W J,HUANG L H,LIANG J B,et al.Progressively Hybrid Transformer for Multi-Modal Vehicle Re-Identification.[J].Sensors,2023,23(9):4206.
[7] LIU X C,LIU W,MEI T,et al.PROVID:Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance[J].IEEE Transactions on Multimedia,2018,20(3):645-658.
[8] ZAKRI A,DENG J,CAI J,et al.Visual Features with Spatio-Temporal-Based Fusion Model for Cross-Dataset Vehicle Re-Identification[J].Electronics,2020,9(7):1099.
[9] ZAKRI A,JIANHUA D,YANG H,et al.Trends in Vehicle Re-Identification Past,Present,and Future:A Comprehensive Review[J].Mathematics,2021,9(24):3162.
[10] HOWARD A G,ZHU M,CHEN B,etal.MobileNets:Efficient Convolutional Neural Networks for Mobile Vision Applications[J].arXiv:1704.04861,2017.
[11] ZHANG X,ZHOU X,LIN M,et al.ShuffleNet:An Extremely Efficient Convolutional Neural Network for Mobile Devices[J].arXiv:1707.01083,2017.
[12] TAN M,LE V Q.EfficientNet:Rethinking Model Scaling for Convolutional Neural Networks[J].arXiv:1905.11946,2019.
[13] TRAN D,WANG H,TORRESANI L,etal.A Closer Look at Spatiotemporal Convolutions for Action Recognition[J].arXiv:1711.11248,2017.
[14] LU J,YANG J,BATRA D,et al.Hierarchical Question-Image Co-Attention for Visual Question Answering[J].arXiv:1606.00061,2016.
[15] ZHAO Q,QU J W.An improved SAM-ShuffleNet model for Ming blue-and-white dragon pattern breakage[J].Modern Electronic Technology,2025,48(9):116-123.
[16] LI J,MA Y,JI Y,et al.SR-SqueezeNet:A lightweight hyperspectral identification model for oil spills based on smoothed activation functions[J].MarinePollution Bulletin,2025,211:117365.
[17] DU Y,GAO C,CHEN X,et al.Mobile malware detection me-thod using improved GhostNetV2 with image enhancement technique.[J].Scientific Reports,2025,15(1):25019.
[18] SUN W,HU Y H,DAI G Z,et al.Component-coupled Transformer network for vehicle re-identification[J].Journal of Computer-Aided Design and Graphics,2023,35(8):1289-1298.
[19] CHEN K I,WANG C M,WANG C M,et al.A vehicle re-identification strategy based on cross-domain two-branch adversarial network[J].Mechanical Design and Manufacturing,2024(4):43-50.
[20] XU X,SONG Y Q,YU K W.Multi-target tracking algorithm based on memory storage[J].Foreign Electronic Measurement Technology,2022,41(3):20-25.
[21] ZHANG L,WU X F,ZHANG S F,et al.Microstructure optimisation study of multi-branch collaborative OSNet[J].Signal Processing,2020,36(8):1335-1343.
[22] HE S,LUO H,WANG P,et al.TransReID:Transformer-based object re-identification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2021:15013-15022.
[23] SUN X,JIN L,WANG H,et al.Spatial awareness enhancement based single-stage anchor-free 3D object detection for autonomous driving[J].Displays,2024,851:02821.
[24] HAN Y,LIN X,PAN D,et al.Pedestrian trajectory predictionmethod based on the Social-LSTM model for vehicle collision[J].Transportation Safety and Environment,2024,6(3):158-169.
[25] ZHU J C,HE H,YANG S L,et al.C3D video fire smoke classification method based on the optimisation of dual attention mechanism[J].Modern Electronic Technology,2025,48(5):53-58.
[26] ZHU L Q,MA S J,LIU M J,et al.A vehicle trajectory prediction method based on Social GAN network with self-attention mechanism[J].Automotive Technology,2025(6):8-14.
[27] WANG D,TANG K,ZENG J,et al.MM-Transformer:ATransformer-Based Knowledge Graph Link Prediction Model That Fuses Multimodal Features[J].Symmetry,2024,16(8):961.
[28] RANI M,KUMAR M.Efficient Human Activity RecognitionUsing a Hybrid MobileNetV2-CNN Model[J].National Academy Science Letters,2025,48(6):1-8.
[29] LIU W,FENG J,ZHU X,et al.Real-time mechanical specific energy prediction for optimizing the drilling parameters using CNN-LSTM model[J].Petroleum Science and Technology,2025,43(18):2413-2436.
[30] XU Z,XU Y,LI Z,et al.I2R:Intra and inter-modal representation learning for code search[J].Intelligent Data Analysis,2024,28(3):807-823.
[31] XU S,YU Y,CHEN Q.Adaptive multimodal feature fusion with frequency domain gate for remote sensing object detection[J].Remote Sensing Letters,2024,15(2):133-144.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发