Abstract: With the rapid development of computer technology,cloud computing technology has become one of the best ways to solve users’ storage and computing power demands.Among them,dynamic virtual machine scheduling based on NUMA architecture has become a hot topic in academia and industry.However,in current research,heuristic algorithms are difficult to schedule virtual machines in real time,and most of the literatures do not consider the energy consumption caused by virtual machine sche-duling under NUMA architecture.This paper proposes a service migration framework of large-scale mobile cloud center virtual machine based on deep reinforcement learning,and constructs the energy consumption model under NUMA architecture.Hierarchical adaptive sampling soft actor critic(HASAC) is proposed.In the cloud computing scenario,the proposed algorithm is compared with the classical deep reinforcement learning methods.Experiment results show that the improved algorithm proposed in this paper can handle more user requests in different scenarios,and consumes less energy.In addition,experiments on various strategies in the algorithm prove the effectiveness of the proposed strategy.

Key words: NUMA architecture, Deep learning, Reinforcement learning, Energy perception, Layered buffer