Abstract: Virtualization technology has become the core support for the rapid development of cloud computing.As a popular distributed framework in cloud environments,the performance of the Hadoop cluster is usually limited by the low efficiency of resource management.With the increasing data volume and cluster scale,it is challenging to efficiently optimize Virtual Machine(VM) placement in the Hadoop cluster to reduce energy consumption,increase resource utilization,and lessen file access latency.To address this important challenge,this paper proposes a novel Multi-objective Optimization with Variable Length Double chromosome(MO-VLD) method for VM placement in the large-scale Hadoop cluster.Firstly,a double chromosome structure is designed by combining the variable length chromosome with NSGA-III.Next,two-stage crossover and mutation operations are introduced to enhance the exploration diversity of solution space.Using the real-world runtime datasets of the Google cluster,extensive simulation experiments demonstrate that the proposed MO-VLD method can effectively handle the dynamic resource demands and improve the resource management efficiency of the Hadoop cluster.Compared to benchmark methods,the MO-VLD method shows superior performance in terms of energy consumption,resource utilization,and file access latency.

Key words: Cloud computing, Hadoop, Virtual machine placement, Multi-objective optimization, Genetic algorithm