Computer Science ›› 2026, Vol. 53 ›› Issue (2): 180-186.doi: 10.11896/jsjkx.250100113

• Database & Big Data & Data Science • Previous Articles     Next Articles

Time Series Forecasting Model Integrating Multi-scale Features and Attention Mechanism

PAN Jian1,2, WANG Xuhao2   

  1. 1 Zhijiang College of Zhejiang University of Technology,Shaoxing,Zhejiang 312030,China
    2 College of Computer Science and Technology,Zhejiang University of Technology,Hangzhou 310023,China
  • Received:2025-01-17 Revised:2025-05-06 Published:2026-02-10
  • About author:PAN Jian,born in 1976,Ph.D,associate professor,master’s supervisor,is a member of CCF(No.26947M).His main research interests include natural language processing,intelligent information processing and Internet of Things.
  • Supported by:
    Natural Science Foundation of Zhejiang Province,China(LGF20F020015).

Abstract: Currently,in the research of time series forecasting tasks,Transformer-based models primarily focus on extracting global and local features from time series data and improving attention mechanisms to reduce model complexity.However,exis-ting methods often overlook the different granularity features exhibited by time series at multiple scales.To address this issue,this paper proposes a time series forecasting model that integrates multi-scale features and the attention mechanism,called MTSformer.Firstly,by down-sampling the original sequence,multiple scale subsequences are obtained,enabling the model to integrate multi-scale feature information and enhance generalization ability.Then,a multi-prediction head structure is used to replace the traditional decoder,which improves prediction speed while reducing model complexity.Finally,experiments are conducted on five benchmark datasets,and the results show that compared with existing methods,the MTSformer model achieves ave-rage reductions of 24.51% in MSE and 17.84% in MAE for time series forecasting.

Key words: Time series forecasting, Multi-scale features, Transformer, Multi-head prediction, Downsampling

CLC Number: 

  • TP391
[1]MAO Y H,SUN C C,XU L Y,et al.A Review of Time Series Forecasting Methods Based on Deep Learning[J].Microelectronics & Computer,2023,40(4):8-17.
[2]CHENG W H,CHE W G.Research on Financial Time SeriesForecasting Algorithm Based on Secondary Decomposition and LSTM[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2022,34(4):638-645.
[3]KAN G Y,YANG J.Research on Meteorological Time Series Forecasting Based on Wavelet Transform and LSTM Hybrid Model[J].Computer Science and Application,2022,12(3):682-689.
[4]XU S,LIU D D.Power Load Forecasting Based on Time Series Combination Model[J].Electronic Design Engineering,2023,31(23):1-6.
[5]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the 31st InternationalConfe-rence on Neural Information Processing Systems.2017:6000-6010.
[6]ZAREMBA W,ILYA S,ORIOL V.Recurrent neural network regularization[J].arXiv:1409.2329,2014.
[7]CHUNG J,GULCEHRE C,CHO K H,et al.Empirical evalu-ation of gated recurrent neural networks on sequence modeling[J].arXiv:1412.3555,2014.
[8]HOCHREITER S,SCHMIDHUBER J.Long short-termmemory[J].Neural Computation,1997,9(8):1735-1780.
[9]LEA C,FLYNN M D,VIDAL R,et al.Temporal convolutional networks for action segmentation and detection[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:156-165.
[10]WU H,HU T,LIU Y,et al.Timesnet:Temporal 2d-variation modeling for general time series analysis[C]//The Eleventh International Conference on Learning Representations.2022.
[11]ZHOU H,ZHANG S,PENG J,et al.Informer:Beyond efficient transformer for long sequence time-series forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:11106-11115.
[12]WU H,XU J,WANG J,et al.Autoformer:Decomposition transformers with auto-correlation for long-term series forecasting[J].Advances in Neural Information Processing Systems,2021,34:22419-22430.
[13]KITAEV N,KAISER Ł,LEVSKAYA A.Reformer:The efficient transformer[J].arXiv:2001.04451,2020.
[14]ZHOU T,MA Z,WEN Q,et al.Fedformer:Frequency enhanced decomposed transformer for long-term series forecasting[C]//International Conference on Machine Learning.PMLR,2022:27268-27286.
[15]NIE Y,NGUYEN N H,SINTHONG P,et al.A time series is worth 64 words:Long-term forecasting with transformers[J].arXiv:2211.14730,2022.
[16]CHEN P,ZHANG Y,CHENG Y,et al.Pathformer:Multi-scale transformers with adaptive pathways for time series forecasting[J].arXiv:2402.05956,2024.
[17]ZENG A L,CHEN M X,ZHANG L,et al.Are transformers effective for time series forecasting?[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023.
[18]ASUNCION A,NEWMAN D.UCI machine learning repository[EB/OL].https://github.com/uci-ml-repo.
[19]CHOLLET F.Deep learning with Python[M]//Manning Publications.2021:45-52.
[20]ZHAO L,GKOUNTOUNA O,PFOSER D.Spatial auto-regressive dependency interpretable learning based on spatial topological constraints[J].ACM Transactions on Spatial Algorithms and Systems,2019,5(3):1-28.
[1] LI Jiahao, JING Junchang, XU Qian, LIU Dong. GTKT:Knowledge Tracing Model Integrating Connectivism Learning and Multi-layer TemporalGraph Transformer [J]. Computer Science, 2026, 53(2): 78-88.
[2] GENG Haijun, LI Dongxin. D-LINet:Time Series Forecasting Framework Integrating Dual-linear Layersand Dual Normalization [J]. Computer Science, 2026, 53(2): 170-179.
[3] LIU Chenhong, LI Fenglian, YANG Jia, WANG Suzhe, CHEN Guijun. Boundary-focused Multi-scale Feature Fusion Network for Stroke Lesion Segmentation [J]. Computer Science, 2026, 53(2): 264-272.
[4] WANG Cheng, JIN Cheng. KAN-based Unsupervised Multivariate Time Series Anomaly Detection Network [J]. Computer Science, 2026, 53(1): 89-96.
[5] DENG Jiayan, TIAN Shirui, LIU Xiangli, OUYANG Hongwei, JIAO Yunjia, DUAN Mingxing. Trajectory Prediction Method Based on Multi-stage Pedestrian Feature Mining [J]. Computer Science, 2025, 52(9): 241-248.
[6] HU Hailong, XU Xiangwei, LI Yaqian. Drug Combination Recommendation Model Based on Dynamic Disease Modeling [J]. Computer Science, 2025, 52(9): 96-105.
[7] DING Zhengze, NIE Rencan, LI Jintao, SU Huaping, XU Hang. MTFuse:An Infrared and Visible Image Fusion Network Based on Mamba and Transformer [J]. Computer Science, 2025, 52(8): 188-194.
[8] LIU Huayong, XU Minghui. Hash Image Retrieval Based on Mixed Attention and Polarization Asymmetric Loss [J]. Computer Science, 2025, 52(8): 204-213.
[9] SHEN Tao, ZHANG Xiuzai, XU Dai. Improved RT-DETR Algorithm for Small Object Detection in Remote Sensing Images [J]. Computer Science, 2025, 52(8): 214-221.
[10] LIU Yajun, JI Qingge. Pedestrian Trajectory Prediction Based on Motion Patterns and Time-Frequency Domain Fusion [J]. Computer Science, 2025, 52(7): 92-102.
[11] HUANG Xingyu, WANG Lihui, TANG Kun, CHENG Xinyu, ZHANG Jian, YE Chen. EFormer:Efficient Transformer for Medical Image Registration Based on Frequency Division and Board Attention [J]. Computer Science, 2025, 52(7): 151-160.
[12] WANG Youkang, CHENG Chunling. Multimodal Sentiment Analysis Model Based on Cross-modal Unidirectional Weighting [J]. Computer Science, 2025, 52(7): 226-232.
[13] LI Yang, LIU Yi, LI Hao, ZHANG Gang, XU Mingfeng, HAO Chongqing. Human Pose Estimation Using Millimeter Wave Radar Based on Transformer and PointNet++ [J]. Computer Science, 2025, 52(6A): 240400169-9.
[14] WANG Xuejian, WANG Yiheng, SUN Xinpo, LIU Chuan, JIA Ming, ZHAO Chao, YANG Chao. Extraction of Crustal Deformation Anomalies Based on Transformer-Isolation Forest [J]. Computer Science, 2025, 52(6A): 240600155-6.
[15] PIAO Mingjie, ZHANG Dongdong, LU Hu, LI Rupeng, GE Xiaoli. Study on Multi-agent Supply Chain Inventory Management Method Based on Improved Transformer [J]. Computer Science, 2025, 52(6A): 240500054-10.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!