计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (12): 81-91.doi: 10.11896/jsjkx.250100030

• 数据库&大数据&数据科学 • 上一篇    下一篇

基于跨模态融合和多生成器的热带气旋预测

刘倩1, 孙虎1, 归耀城2, 周国强1,3   

  1. 1 南京邮电大学计算机学院、软件学院、网络空间安全学院 南京 210023
    2 南京邮电大学现代邮政学院 南京 210023
    3 地球信息工程国家重点实验室 西安 710018
  • 收稿日期:2025-01-06 修回日期:2025-04-25 出版日期:2025-12-15 发布日期:2025-12-09
  • 通讯作者: 周国强(zhougq@njupt.edu.cn)
  • 作者简介:(qianliu@njupt.edu.cn)
  • 基金资助:
    国家重点基础研究计划(2020YFA0713600);国家自然科学基金(62272214);国家对地观测科学数据中心开放课题(NODAOP2024011)

Tropical Cyclone Forecasting Based on Cross-modal Fusion and Multi-generators

LIU Qian1, SUN Hu1, GUI Yaocheng2, ZHOU Guoqiang1,3   

  1. 1 School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
    2 School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
    3 State Key Laboratory of Geo-Information Engineering, Xi’an 710018, China
  • Received:2025-01-06 Revised:2025-04-25 Published:2025-12-15 Online:2025-12-09
  • About author:LIU Qian,born in 1986,Ph.D,lecturer,is a member of CCF(No.98989M).Her main research interests include artificial intelligence and sentiment analysis.
    ZHOU Guoqiang,born in 1968,Ph.D,associate professor.His main research interests include machine learning,distributed computing and data analysis.
  • Supported by:
    This work was supported by the National Key Basic Research Program (2020YFA0713600),National Natural Science Foundation of China(62272214) and National Earth Observation Data Center Open Project(NODAOP2024011).

PDF (PC)

15

摘要： 准确预测热带气旋的运动轨迹和强度对减轻和预防灾害至关重要。基于深度学习的方法虽然表现出出色的预测性能,但这类方法大多只关注单模态数据,忽略了不同模态之间的相关性。为了充分利用多模态数据中的丰富信息,提出一种基于跨模态融合和多生成器的热带气旋预测模型。该模型包括一个多模态特征提取模块、一个跨模态特征融合模块和一个生成对抗网络。多模态特征提取模块从历史最佳轨迹数据、大气再分析数据以及环境场数据中分别提取相应的特征表示。跨模态特征融合模块通过一种新颖的跨模态特征互补策略融合多模态特征。生成对抗网络通过多个生成器生成最终的热带气旋预测结果。此外,还构建了一个特征融合损失以帮助提高模型的性能。实验表明,所提方法不仅能在训练和推理阶段都保持较高的效率,而且具有更好的预测性能。

关键词: 热带气旋预测, 跨模态融合, 多生成器, 注意力机制

Abstract: Accurately predicting the trajectory and intensity of tropical cyclones(TCs) is essential for disaster mitigation and prevention.Although deep learning-based advances have demonstrated great prediction performance,a majority of these methods only focus on unimodal data and overlook the rich correlations between different modalities.To fully leverage the rich information contained in multimodal data,a novel framework for tropical cyclone forecasting based on cross-modal fusion and multi-generators is proposed.The framework includes a multimodal feature extraction module,a cross-model feature fusion module and a generative adversarial network(GAN).The multimodal feature extraction module obtains feature representations from the best historical trajectory data,atmospheric reanalysis data and environmental field data respectively.The cross-model feature fusion module fuses multimodal features through a novel cross-modal feature complementation strategy.GAN produces final TC predictions through multiple generators.Additionally,a feature fusion loss is constructed to help boosting the model’s performance.Experiments show that the proposed method can not only maintain high efficiency in training and inference stages,but also achieve better prediction performance.

Key words: Tropical cyclone prediction, Cross-modal fusion, Multiple generators, Attention mechanism

中图分类号: 

  • TP391.1
[1]WANG Y,LI Y,XU J,et al.The intensity dependence of tropical cyclone intensification rate in a simplified energetically based dynamical system model[J].Journal of the Atmospheric Sciences,2021,78(7):2033-2045.
[2]JING R,LIN N,EMANUEL K,et al.A comparison of tropical cyclone projections in a high-resolution global climate model and from downscaling by statistical and statistical-deterministic methods [J].Journal of Climate,2021,34(23):9349-9364.
[3]KIM S H,AHN J B,SUN J.Dynamical-statistical long-termprediction for tropical cyclone landfalls in east asia [J].International Journal of Climatology,2022,42(4):2586-2600.
[4]LIU Y,RASOULI S,WONG M,et al.Rt-gcn:Gaussian-based spatiotemporal graph convolutional network for robust traffic prediction [J].Information Fusion,2024,102:102078.
[5]JIANG Z,ZHOU Z,LI L,et al.Back to optimization:Diffusion-based zero-shot 3d human pose estimation[C]//IEEE/CVF Winter Conference on Applications of Computer Vision.2024:6142-6152.
[6]ALEMANY S,BELTRAN J,PEREZ A,et al.Predicting hurricane trajectories using a recurrent neural network [C]//AAAI Conference on Artificial Intelligence.2019:468-475.
[7]GAO S,ZHAO P,PAN B,et al.A nowcasting model for the prediction of typhoon tracks based on a long short term memory neural network [J].Acta Oceanologica Sinica,2018,37:8-12.
[8]FARMANIFARD S,ALESHEIKH A A,SHARIF M.A context-aware hybrid deep learning model for the prediction of tropical cyclone trajectories [J].Expert Systems with Applications,2023,231:120701.
[9]WANG C,XU Q,LI X,et al.CNN-based tropical cyclone track forecasting from satellite infrared images[C]//IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium.IEEE,2020:5811-5814.
[10]ZHANG R,LIU Q,HANG R.Tropical cyclone intensity estimation using two-branch convolutional neural network from infrared and water vapor images [J].IEEE Transactions on Geoscience and Remote Sensing,2019,58(1):586-597.
[11]ZHANG Z,YANG X,WANG X,et al.A neural network with spatiotemporal encoding module for tropical cyclone intensity estimation from infrared satellite image [J].Knowledge-Based Systems,2022,258:110005.
[12]JIANG W,HU G,WU T,et al.Dmanet kf:Tropical cyclone intensity estimation based on deep learning and kalman filter from multispectral infrared images [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2023,16:4469-4483.
[13]MA Z,YAN Y,LIN J,et al.A multiscale and multilayer feature extraction network with dual attention for tropical cyclone intensity estimation [J].IEEE Transactions on Geoscience and Remote Sensing,2024,62:1-15.
[14]NGIAM J,KHOSLA A,KIM M,et al.Multimodal deep learning [C]//The 28th International Conference on Machine Learning.2011:689-696.
[15]WANG X,CHEN K,LIU L,et al.Global tropical cyclone intensity forecasting with multi-modal multi-scale causal autoregressive model.[J].arXiv:2402.13270,2024.
[16]JIANG S,FAN H,WANG C.Improvement of typhoon intensity forecasting by using a novel spatio-temporal deep learning model [J].Remote Sensing,2022,14(20):5205.
[17]MA D,WANG L,FANG S,et al.Tropical cyclone intensity prediction by interand intra-pattern fusion based on multi-source data [J].Environmental Research Letters,2023,18(1):014020.
[18]QIN W,TANG J,LU C,et al.A typhoon trajectory prediction model based on multimodal and multitask learning [J].Applied Soft Computing,2022,122:108804.
[19]LIU Z,HAO K,GENG X,et al.Dual-branched spatio-temporal fusion network for multihorizon tropical cyclone track forecast [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2022,15:3842-3852.
[20]XU G,LIN K,LI X,et al.Saf-net:A spatio-temporal deep lear-ning method for typhoon intensity prediction [J].Pattern Recognition Letters,2022,155:121-127.
[21]TIAN W,ZHOU X,NIU X,et al.A lightweight multitasklearning model with adaptive loss balance for tropical cyclone intensity and size estimation [J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2022,16:1057-1071.
[22]GAN S,FU J,ZHAO G,et al.Short-term prediction of tropical cyclone track and intensity via four mainstream deep learning techniques [J].Journal of Wind Engineering and Industrial Aerodynamics,2024,244:105633.
[23]NA Y,NA B,SON S.Near real-time predictions of tropical cyclone trajectory and intensity in the northwestern pacific ocean using echo state network [J].Climate Dynamics,2022,58(3/4):651-667.
[24]TONG B,WANG X,FU J,et al.Short-term prediction of the intensity and track of tropical cyclone via convlstm model [J].Journal of Wind Engineering and Industrial Aerodynamics,2022,226:105026.
[25]SHI X,CHEN Z,WANG H,et al.Convolutional lstm network:a machine learning approach for precipitation nowcasting [C]//The 28th International Conference on Neural Information Processing Systems.2015:802-810.
[26]HUANG C,BAI C,CHAN S,et al.Mmstn:A multi-modal spatial-temporal network for tropical cyclone short-term prediction [J].Geophysical Research Letters,2022,49(4):1-11.
[27]DENDORFER P,ELFLEIN S,LEAL-TAIX E L.Mg-gan:Amulti-generatormodel preventing out-of-distribution samples in pedestrian trajectory prediction [C]//IEEE/CVF International Conference on Computer Vision.2021:13158-13167.
[28]HUANG C,BAI C,CHAN S,et al.Mgtcf:multi-generator tropical cyclone forecasting with heterogeneous meteorological data [C]//AAAI Conference on Artificial Intelligence.2023:5096-5104.
[29]YING M,ZHANG W,YU H,et al.An overview of the china meteorological administration tropical cyclone database [J].Journal of Atmospheric and Oceanic Technology,2014,31(2):287-301.
[30]ÇIÇEK O,ABDULKADIR A,LIENKAMP S S,et al.3D U-Net:learning dense volumetric segmentation from sparse annotation [C]//The 19th International Conference on Medical Image Computing and Computer-Assisted Intervention:Part II.2016:424-432.
[31]AGRIM G,JUSTIN J,LI F,et al.Social gan:Socially acceptable trajectories with generative adversarial networks [C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:2255-2264.
[32]GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al.Ge-nerative adversarial nets[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems.Cambridge,MA:MIT,2014:2672-2680.
[33]KINGMA D P,BA J.Adam:A method for stochastic optimization [C]//The 3rd International Conference on Learning Representations.2015.
[34]HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[35]CHO K,MERRIENBOER B V,GULCEHRE C,et al.Learning phrase representations using rnn encoder-decoder for statistical machine translation [C]//The 2014 Conference on Empirical Methods in Natural Language Processing.2014:1724-1734.
[36]PAN B,XU X,SHI Z.Tropical cyclone intensity predictionbased on recurrent neural networks[J].Electronics Letters,2019,55(7):413-415.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发