Abstract: With the rapid development of Internet technology,Web log mining based on a single node becomes very difficult.The emergence of Hadoop cloud platform provides a new solution to this problem.However,the traditional Web log mining clustering algorithm k-means is sensitive to the initial cluster centers selection,so it will easily affect the accuracy of clustering.Thus for this problem,this paper proposed a k-means algorithm based on particle swarm optimization which makes the k-means algorithm not be affected by the initial cluster centers.And the algorithm is realized in the Hadoop MapReduce programming platform.Experimental results show that: compared with traditional k-means algorithm the proposed algorithm has the higher clustering accuracy,and compared with stand-alone serial algorithm, the operating efficiency improved greatly.

Key words: Hadoop,k-means,PSO,MapReduce,Web log mining