Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (9): 178-185.doi: 10.11896/jsjkx.241100174

• High Performance Computing • Previous Articles     Next Articles

Prediction of Resource Usage on High-performance Computing Platforms Based on ARIMAand LSTM

LI Siqi, YU Kun, CHEN Yuhao   

  1. School of Data Science and Engineering,East China Normal University,Shanghai 200062,China
  • Received:2024-11-28 Revised:2025-01-26 Online:2025-09-15 Published:2025-09-11
  • About author:LI Siqi,born in 2002,postgraduate,is a member of CCF(No.V2924G).Her main research interests include high-performance computing and data analysis.
    YU Kun,born in 1979,master,engineer,is a member of CCF(No.I8213M).Her main research interests include high-performance computing,computational resource management and optimization.
  • Supported by:
    National Natural Science Foundation of China(62377013).

PDF (PC)

11

Abstract: The rapid increase in data scale and experimental complexity in scientific research and engineering simulations has significantly heightened the demand for high-performance computing(HPC) resources.However,limited availability of such resources necessitates efficient utilization strategies.This paper analyzes over 400 000 job records collected from the HPC cluster at East China Normal University between January 2022 and November 2023.After preprocessing the data,daily job counts and CPU utilization rates are extracted to represent the cluster’s resource usage patterns.Autoregressive Integrated Moving Average(ARIMA),Long Short-Term Memory with Dual Dropouts(2DLSTM),and a hybrid ARIMA-2DLSTM are applied to fit the historical data and forecast short-term and long-term resource usage.Model performance is evaluated by mean absolute error(MAE) and mean squared error(MSE) metrics.Results indicate that the ARIMA-2DLSTM hybrid model achieves superior predictive accuracy compared to standalone ARIMA and 2DLSTM models.The hybrid model effectively captures trend changes of the cluster’s resource usage patterns and accurately predicts peak and trough timings,providing critical insights for optimizing resource allocation in HPC environments.

Key words: High-performance computing, Time series analysis, ARIMA model, LSTM model, ARIMA-LSTM hybrid model

CLC Number: 

  • TP183
[1]LU P J,XIONG Z Y,LAI M C.Survey on High-performanceComputing Technology and Standards[J].Computer Science,2023,50(11):1-7.
[2]Editorial Department of this Journal.Top 10 Frontiers ofScience and Technology in 2023[J].Science and Technology Think Tank,2023(1):1-4.
[3]WANG J Y,ZHOU B Y,ZHANG F,et al.Overview of Energy Consumption Models and Energy Efficiency Algorithms in Data Centers[J].Journal of Computer Research and Development,2019,56(8):1587-1603.
[4]CHEN Y J.Energy Consumption Analysis and Power Consump-tion Prediction of High-Performance Computing Clusters[D].Jinan:Qilu University of Technology,2023.
[5]SUN J W.Data-Driven Research on Execution Time Predictionand Optimization for High-Performance Computing Programs[D].Hefei:University of Science and Technology of China,2020.
[6]ZHANG J P.Research on Active Learning Methods for Empirical Performance Modeling of High-Performance Computing Programs[D].Hefei:University of Science and Technology of China,2020.
[7]KUMAR A S,MAZUMDAR S.Forecasting HPC workloadusing ARMA models and SSA[C]//2016 International Confe-rence on Information Technology(ICIT).IEEE,2016:294-297.
[8]BI J,ZHANG L,YUAN H,et al.Hybrid task prediction based on wavelet decomposition and ARIMA model in cloud data center[C]//2018 IEEE 15th International Conference on Networking,Sensing and Control(ICNSC).IEEE,2018:1-6.
[9]IPEK E,DE SUPINSKI B R,SCHULZ M,et al.An approach to performance prediction for parallel applications[C]//Euro-Par 2005 Parallel Processing:11th International Euro-Par Confe-rence,Lisbon,Portugal,August 30-September 2,2005.Procee-dings 11.Berlin:Springer,2005:196-205.
[10]BORGHESI A,BARTOLINI A,LOMBARDI M,et al.Predictive modeling for job power consumption in HPC systems[C]//High Performance Computing:31st International Conference,ISC High Performance 2016,Frankfurt,Germany,June 19-23,2016,Proceedings.Springer,2016:181-199.
[11]KONTOPOULOU V I,PANAGOPOULOS A D,KAKKOS I,et al.A review of ARIMA vs.machine learning approaches for time series forecasting in data driven networks[J].Future Internet,2023,15(8):255.
[12]SHARIFF S M.Autoregressive Integrated Moving Average(ARIMA) and Long Short-Term Memory(LSTM) network models for forecasting energy consumptions[J].European Journal of Electrical Engineering and Computer Science,2022,6(3):7-10.
[13]SIRISHA U M,BELAVAGI M C,ATTIGERI G.Profit prediction using ARIMA,SARIMA and LSTM models in time series forecasting:A comparison[J].IEEE Access,2022,10:124715-124727.
[14]FAN D,SUN H,YAO J,et al.Well production forecastingbased on ARIMA-LSTM model considering manual operations[J].Energy,2021,220:119708.
[15]SELVIN S,VINAYAKUMAR R,GOPALAKRISHNAN E A,et al.Stock price prediction using LSTM,RNN and CNN-sliding window model[C]//2017 International Conference on Advances in Computing,Communications and Informatics(ICACCI).IEEE,2017:1643-1647.
[16]SCHAFFER A L,DOBBINS T A,PEARSON S A.Interrupted time series analysis using autoregressive integrated moving average(ARIMA) models:a guide for evaluating large-scale health interventions[J].BMC Medical Research Methodology,2021,21:1-12.
[17]BOX G E P,JENKINS G M,REINSEL G C,et al.Time series analysis:forecasting and control[M].John Wiley & Sons,2015:88-126.
[18]ARUNKUMAR K E,KALAGA D V,KUMAR C M S,et al.Forecasting the dynamics of cumulative COVID-19 cases(confirmed,recovered and deaths) for top-16 countries using statistical machine learning models:Auto-Regressive Integrated Mo-ving Average(ARIMA) and Seasonal Auto-Regressive Integrated Moving Average(SARIMA)[J].Applied Soft Computing,2021,103:107161.
[19]HOCHREITER S,SCHMIDHUBER J.Long Short-term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[20]CHOUDAKKANAVAR G,MANGAI J A.A hybrid 1D-CNN-Bi-LSTM based model with spatial dropout for multiple fault diagnosis of roller bearing[J].International Journal of Advanced Computer Science and Applications,2022,13(8):0130873.
[1] GAO Yiqin, LUO Zhiyu, WANG Yichao, LIN Xinhua. Performance Evaluation and Optimization of Operating System for Domestic Supercomputer [J]. Computer Science, 2025, 52(5): 11-24.
[2] SHANG Qiuyan, LI Yicong, WEN Ruilin, MA Yinping, OUYANG Rongbin, FAN Chun. Two-stage Multi-factor Algorithm for Job Runtime Prediction Based on Usage Characteristics [J]. Computer Science, 2025, 52(2): 261-267.
[3] LUO Haiwen, WU Yangjun, SHANG Honghui. Many-core Optimization Method for the Calculation of Ab initio Polarizability [J]. Computer Science, 2023, 50(6): 1-9.
[4] LU Pingjing, XIONG Zeyu, LAI Mingche. Survey on High-performance Computing Technology and Standards [J]. Computer Science, 2023, 50(11): 1-7.
[5] LI Hao-dong, HU Jie, FAN Qin-qin. Multimodal Multi-objective Optimization Based on Parallel Zoning Search and Its Application [J]. Computer Science, 2022, 49(5): 212-220.
[6] LI Zhi-ying, MA Shuo, ZHOU Chao, MA Ying-jin, LIU Qian, JIN Zhong. “AI+HPC”-based Time Prediction for the First Principle Calculations and Its Applications in Biomed Community [J]. Computer Science, 2022, 49(10): 36-43.
[7] HUANG Ming, SUN Lin-fu, REN Chun-hua , WU Qi-shi. Improved KNN Time Series Analysis Method [J]. Computer Science, 2021, 48(6): 71-78.
[8] WANG You-wei, ZHU Chen, ZHU Jian-ming, LI Yang, FENG Li-zhou, LIU Jiang-chun. User Interest Dictionary and LSTM Based Method for Personalized Emotion Classification [J]. Computer Science, 2021, 48(11A): 251-257.
[9] XU He, WU Man-xing, LI Peng. RFID Indoor Relative Position Positioning Algorithm Based on ARIMA Model [J]. Computer Science, 2020, 47(9): 252-257.
[10] CHEN Pei, ZHENG Wan-bo, LIU Wen-qi, XIAO Min, ZHANG Ling-xiao. Analysis and Forecast of Some Climate Indexes in Main Producing Areas of Yunnan Province Based on Multiple Models [J]. Computer Science, 2020, 47(11A): 496-503.
[11] HUANG Qiu-lan, LI Hai-bo, SHI Jing-yan, SUN Zhen-yu, WU Wen-jing, CHENG Yao-dong and CHENG Zhen-jing. Openstack-based Virtualized Computing Cluster and Application for High Energy Physics [J]. Computer Science, 2017, 44(10): 59-63.
[12] ZHANG Yan-qing, LU Yu-liang and YANG Guo-zheng. Measurement of AS-level Internet Evolution Based on Temporal Distance [J]. Computer Science, 2016, 43(8): 118-122.
[13] LAI Ji-bao,MENG Yuan,YU Tao,WANG Yu-jing,LIN Ying-hao and LV Tian-ran. Research on Cubic Convolution Interpolation Parallel Algorithm Based on Dual-GPU [J]. Computer Science, 2013, 40(8): 24-27.
[14] . Research of Job Scheduling Strategy of High-performance Computer Based on Adaptive Power Management [J]. Computer Science, 2012, 39(10): 313-317.
[15] . [J]. Computer Science, 2009, 36(3): 21-25.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.