Abstract: Join operation is the most common operation in data analysis applications with large-scale datasets,and Map-Reduce can not support join operation perfectly in handling data skew problem.MapReduce frequecncy classified join algorithm was proposed,and datasets were classified into three categories according to the appeared data frequency.Data redistribution applying partitioning algorithm and broadcast algorithms eliminate the impact of skewed data.And data redistribution is realized by using hash algorithm for the non-skew data.Join operation can be completed in a single node,avoiding the cost of communications across the nodes under the MapReduce for the redistributed data,and balancing the workload of each node effectively,thereby improves the efficiency of join operations in skewed data.The effectiveness and practicality of the algorithms are proved by the comparison with traditional algorithms.

Key words: Data skew,MapReduce,Join algorithm,Load balancing