Abstract: The MapReduce task load balancing in Hadoop mainly depends on the partition function. The Hadoop default partition function is not efficient in practical business processing. This paper presented a load balancing strategy based on the weight value of the periodic jobs. Because the data's distribution is similar in each period, we calculated the weight from historical data's profile. Through analyzing a sample data in Map phase to predict the whole data weighted integral approximate distribution, the strategy guids the Reduce partition to ensure its load balancing. We also presented the difference between TeraSort strategy and the new strategy. The experimental results with the view video logs show that the performance of our strategy is improved about 2 times compared with the default strategy.

Key words: MapReduce, TeraSort, Load balance, Periodic