基于深度强化学习的数据中心热感知能耗优化方法

doi:10.11896/jsjkx.230500109

计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230500109-8.doi: 10.11896/jsjkx.230500109

基于深度强化学习的数据中心热感知能耗优化方法

李丹阳¹, 吴良基¹, 刘慧², 姜静清³

1 东北大学软件学院沈阳 110169
2 东北大学冶金学院沈阳 112000
3 内蒙古民族大学计算机科学与技术学院内蒙古通辽 028000

发布日期:2024-06-06
通讯作者: 吴良基(2171357@stu.neu.edu.cn)
作者简介:(2110497@stu.neu.edu.cn)
基金资助:
国家自然科学基金(62162050)

Deep Reinforcement Learning Based Thermal Awareness Energy Consumption OptimizationMethod for Data Centers

LI Danyang¹, WU Liangji¹, LIU Hui², JIANG Jingqing³

1 Software College,Northeastern University,Shenyang 110169,China
2 School of Metallurgy,Northeastern University,Shenyang 112000,China
3 College of Computer Science and Technology,Inner Mongolia Minzu University,Tongliao,Inner Mongolia 028000,China

Published:2024-06-06
About author:LI Danyang,born in 1997,Ph.D.Her main research interests include green computing and energy saving.
WU Liangji,born in 1999,postgra-duate.His main research interests include green computing and reinforcement learning.
Supported by:
National Natural Science Foundation of China(62162050).

摘要/Abstract

摘要： 随着数据中心规模的不断扩大,所引起的高能耗、高运营成本和环境污染等问题日益严重,严重影响了数据中心的可持续性。大多数数据中心能耗优化方法为了降低计算能耗,会将任务集中在尽可能少的服务器上,但这样做往往会导致数据中心热点的产生,并且提高了冷却能耗。为了解决这一问题,文中首先对数据中心进行建模,并将数据中心总能耗优化问题建模为一个任务调度问题,并且要求调度过程中不产生数据中心热点。为了解决该问题,文中提出了一种基于深度强化学习的数据中心任务调度方法,并使用奖励塑造对该方法进行优化,在不产生热点的前提下降低数据中心的总能耗。最后,通过仿真环境和真实数据中心负载跟踪数据进行了实验。仿真实验结果表明,所提方法相比其他现有的调度方法能够更好地降低数据中心总能耗,最多降低了25.5%。此外,提出的优化方法还不会产生热点,这进一步证明了其优越性。

关键词: 数据中心, 能耗优化, 热点, 任务调度, 深度强化学习, 奖励塑造

Abstract: With the continuous expansion of the scale of data centers,the problems of high energy consumption,high operating costs and environmental pollution are becoming more and more serious,which seriously affect the sustainability of data centers.Most data center energy consumption optimization methods focus tasks on as few servers as possible,so as to reduce computing energy consumption.However,this often leads to the generation of data center hotspots and increases cooling energy consumption.In order to solve this problem,this paper first models the data center,and models the total energy consumption optimization problem of the data center as a task scheduling problem,and requires that no data center hotspots are generated.This paper proposes a task scheduling method based on deep reinforcement learning for data centers,and uses reward shaping to optimize the method to reduce the total energy consumption of data centers without generating hotspots.Finally,experiments are carried out through simulation environment and real data center load trace data.The simulation results show that the proposed method can reduce the total energy consumption of the data center better than other existing scheduling methods,and can reduce the total energy consumption by up to 25.5%.In addition,the proposed optimization method does not generate hot spots yet,which further proves its superiority.

Key words: Data Center, Energy consumption optimization, Hot spot, Task scheduling, Deep reinforcement learning, Reward shaping

中图分类号:

TP181

李丹阳, 吴良基, 刘慧, 姜静清. 基于深度强化学习的数据中心热感知能耗优化方法[J]. 计算机科学, 2024, 51(6A): 230500109-8. https://doi.org/10.11896/jsjkx.230500109

LI Danyang, WU Liangji, LIU Hui, JIANG Jingqing. Deep Reinforcement Learning Based Thermal Awareness Energy Consumption OptimizationMethod for Data Centers[J]. Computer Science, 2024, 51(6A): 230500109-8. https://doi.org/10.11896/jsjkx.230500109

参考文献

[1]NADJAHI C,LOUAHLIA H,LEMASSON S.A review ofthermal management and innovative cooling strategies for data center[J].Sustainable Computing:Informatics and Systems,2018,19:14-28.
[2]DING J,ZHANG H,LENG D,et al.Experimental investigation and application analysis on an integrated system of free cooling and heat recovery for data centers[J].International Journal of Refrigeration,2022,136:142-151.
[3]TENG F,YU L,LI T,et al.Energy efficiency of VM consolida-tion in IaaS clouds[J].The Journal of Supercomputing,2017,73(2):782-809.
[4]KAPLAN J M,FORREST W,KINDLER N.Revolutionizingdata center energy efficiency[J].McKinsey & Company,2008:1-13.
[5]HE K,LI Z,DENG D,et al.Energy-efficient framework for virtual machine consolidation in cloud data centers[J].China Communications,2017,14(10):192-201.
[6]ILAGER S,RAMAMOHANARAO K,BUYYA R.ETAS:Energy and thermal-aware dynamic virtual machine consolidation in cloud data center with proactive hotspot mitigation[J].Concurrency and Computation:Practice and Experience,2019,31(17):e5221.1-e5221.15.
[7]LIN M,WIERMAN A,ANDREW L L H,et al.Dynamic Right-Sizing for Power-Proportional Data Centers[J].IEEE/ACM Transactions on Networking,2013,21(5):1378-1391.
[8]FELLER E,ROHR C,MARGERY D,et al.Energy Management in IaaS Clouds:A Holistic Approach[C]//2012 IEEE Fifth International Conference on Cloud Computing.2012:204-212.
[9]MNIH V,KAVUKCUOGLU K,SILVER D,et al.Playing Atari with Deep Reinforcement Learning[A].arXiv,2013.
[10]KIRAN B R,SOBH I,TALPAERT V,et al.Deep Reinforce-ment Learning for Autonomous Driving:A Survey[J].IEEE Transactions on Intelligent Transportation Systems,2022,23(6):4909-4926.
[11]GU S,HOLLY E,LILLICRAP T,et al.Deep reinforcementlearning for robotic manipulation with asynchronous off-policy updates[C]//2017 IEEE International Conference on Robotics and Automation(ICRA).2017:3389-3396.
[12]WANG Y,LIU H,ZHENG W,et al.Multi-Objective Workflow Scheduling With Deep-Q-Network-Based Multi-Agent Reinforcement Learning[J].IEEE Access,2019,7:39974-39982.
[13]AKBARI A,KHONSARI A,GHOREYSHI S M.Thermal-Aware Virtual Machine Allocation for Heterogeneous Cloud Data Centers[J].Energies,2020,13(11):2880.
[14]LIU H,LIU B,YANG L T,et al.Thermal-Aware and DVFS-Enabled Big Data Task Scheduling for Data Centers[J].IEEE Transactions on Big Data,2018,4(2):177-190.
[15]AGHASI A,JAMSHIDI K,BOHLOOLI A.A thermal-awareenergy-efficient virtual machine placement algorithm based on fuzzy controlled binary gravitational search algorithm(FC-BGSA)[J].Cluster Computing,2022,25(2):1015-1033.
[16]TANG Q,GUPTA S K S,VARSAMOPOULOS G.Energy-Efficient Thermal-Aware Task Scheduling for Homogeneous High-Performance Computing Data Centers:A Cyber-Physical Approach[J].IEEE Transactions on Parallel and Distributed Systems,2008,19(11):1458-1472.
[17]DONG T,XUE F,XIAO C,et al.Task scheduling based on deep reinforcement learning in a cloud manufacturing environment[J].Concurrency and Computation:Practice and Experience,2020,32(11):e5654.
[18]LI F,HU B.Deepjs:Job scheduling based on deep reinforcement learning in cloud data center[C]//Proceedings of the 4th international conference on big data and computing.2019:48-53.
[19]CHENG M,LI J,NAZARIAN S.DRL-cloud:Deep reinforcement learning-based resource provisioning and task scheduling for cloud service providers[C]//2018 23rd Asia and South Pacific Design Automation Conference(ASP-DAC).Jeju:IEEE,2018:129-134.
[20]MAO H,SCHWARZKOPF M,VENKATAKRISHNAN S B,et al.Learning scheduling algorithms for data processing clusters[C]//Proceedings of the ACM Special Interest Group on Data Communication.New York,NY,USA:Association for Computing Machinery,2019:270-288.
[21]SHI L,WEN L,LEI S,et al.Virtual machine consolidation algorithm based on decision tree and improved Q-learning by uniform distribution[J].Computer Science,2023,50(6):36-44.
[22]LU H F,GU C H,LUO F,et al.Virtual machine placement strategy with energy consumption optimization under reinforcement learning[J].Computer Science,2019,46(9):291-297.
[23]BELOGLAZOV A,ABAWAJY J,BUYYA R.Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing[J].Future Generation Computer Systems,2012,28(5):755-768.
[24]MOORE J D,CHASE J S,RANGANATHAN P,et al.Making scheduling “Cool”:Temperature-aware workload placement in data centers[C]//USENIX annual technical conference,general track.2005:61-75.
[25]LILLICRAP T P,HUNT J J,PRITZEL A,et al.Continuouscontrol with deep reinforcement learning[J].arXiv,2019.
[26]PARK K,PAI V S.CoMon:a mostly-scalable monitoring system for PlanetLab[J].ACM SIGOPS Operating Systems Review,2006,40(1):65-74.
[27]LU C,YE K,XU G,et al.Imbalance in the cloud:An analysis on Alibaba cluster trace[C]//2017 IEEE International Conference on Big Data(Big Data).2017:2884-2892.
[28]WANG L,VON LASZEWSKI G,DAYAL J,et al.TowardsThermal Aware Workload Scheduling in a Data Center[C]//2009 10th International Symposium on Pervasive Systems,Algorithms,and Networks.2009:116-122.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

基于深度强化学习的数据中心热感知能耗优化方法

Deep Reinforcement Learning Based Thermal Awareness Energy Consumption OptimizationMethod for Data Centers

PDF (PC)

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 0

Metrics

本文评价

推荐阅读 0