计算机科学 ›› 2024, Vol. 51 ›› Issue (2): 293-299.doi: 10.11896/jsjkx.230100031
王杨民, 胡成玉, 颜雪松, 曾德泽
WANG Yangmin, HU Chengyu, YAN Xuesong, ZENG Deze
摘要： 随着计算机技术的快速发展,云计算技术成为了解决用户存储、算力需求的最佳方法之一。其中,基于NUMA架构的动态虚拟机调度成为了学术界和工业界关注的热点方向。但是,目前的研究中,基于启发式的算法难以对虚拟机进行实时调度,并且大多数文献没有考虑NUMA架构下虚拟机调度产生的能耗等问题。对此,提出了一种基于深度强化学习的大型移动云中心虚拟机服务迁移框架,构建了NUMA架构下的能耗模型;提出了自适应奖励的分层自适应柔性演员评论家算法(Hie-rarchical Adaptive Sampling Soft Actor Critic,HASAC);在云计算场景下,将所提算法与3种经典的深度强化学习方法进行实验对比。实验结果表明,所提改进算法在不同场景下可以处理更多的用户请求,且消耗的能源较少。此外,对算法中各种策略进行消融实验,证明了所提策略的有效性。
中图分类号:
| [1]DUAN W,HU M.Overview of Cloud Computing System Reliability Research[J].Computer Research and Development,2020,57(1):102-123. [2]HE D,CHEN L,LIANG J,et al.NUMA-Aware ContentionScheduling on Multicore Systems [C]//2021 16th International Conference on Intelligent Systems and Knowledge Engineering(ISKE).IEEE,2021:452-457. [3]CHENG Y,CHEN W,WANG Z.Performance-monitoring-based traffic-aware virtual machine deployment on NUMA systems [J].IEEE Systems Journal,2015,11(2):973-982. [4]BISWAI N K,BANERJEE S,BISWAS U.An approach towards development of new linear regression prediction model for reduced energy consumption and SLA violation in the domain of green cloud computing[J].Sustainable Energy Technologies and Assessments,2021,45:101087. [5]MA O,JIANG X,JIN G.Heuristic dynamic threshold algorithm for priority inversion suppression of LEDBAT protocol [J].Computer Research and Development,2020,57(6):1292-1301. [6]REN H,WANG Y,XU C.Smig-rl:An evolutionary migration framework for cloud services based on deep reinforcement lear-ning[J].ACM Transactions on Internet Technology(TOIT),2020,20(4):1-18. [7]KRUEKAEW B,KIMPAN W.Enhancing of artificial bee colony algorithm for virtual machine scheduling and load balancing problem in cloud computing[J].International Journal of Computational Intelligence Systems,2020,13(1):496-510. [8]RAGMANI A,ELOMRI A,ABGHOUR N,et al.FACO:A hybrid fuzzy ant colony optimization algorithm for virtual machine scheduling in high-performance cloud computing[J].Journal of Ambient Intelligence and Humanized Computing,2020,11:3975-3987. [9]TARAFDAR A,KHATUA S,DAS R K.QoS aware energy efficient VM consolidation techniques for a virtualized data center[C]//2018 IEEE/ACM 11th International Conference on Utility and Cloud Computing(UCC).IEEE,2018:114-123. [10]WANG Y,LIU H,ZHENG W,et al.Multi-objective workflow scheduling with deep-Q-network-based multi-agent reinforcement learning[J].IEEE Access,2019,7:39974-39982. [11]ZHANG S,WU T,PAN M,et al.A-SARSA:A predictive container auto-scaling algorithm based on reinforcement learning [C]//2020 IEEE International Conference on Web Services(ICWS).IEEE,2020:489-497. [12]LIU L,XU J,YU H,et al.Joint Admission Control and Provisioning for Virtual Machines[C]//2015 IEEE International Conference on Communications(ICC).London,UK,2015:332-337. [13]HU K,LIN W,HUANG T,et al.Virtual Machine Consolidation for NUMA Systems:A Hybrid Heuristic Grey Wolf Approach [C]//2020 IEEE 26th International Conference on Parallel and Distributed Systems(ICPADS).IEEE,2020:569-576. [14]SHENG J,HU Y,ZHOU W.Learning to schedule multi-NUMA virtual machines via reinforcement learning [J].Pattern Recognition,2022,121:108254. [15]OH Y,KIMIN L,SHIN J.Learning to Sample with Local and Global Contexts in Experience Replay Buffers [C]//International Conference on Learning Representations(ICLR).2021. |
|
||
首页