关键词: 微簇,等价连接,Hadoop平台,聚类

Abstract: Hadoop is a popular platform to handle huge datasets．But many clustering algorithms can not run effectively over it,for it lacks built-in support for iterative programs,which arises naturally in many clustering applications．We proposed bigClustering which can be easily parallelized in Hadoop MapReduce and done in quite a few MapReduce rounds．Our algorithm is based on the ideas of micro-cluster and equivalence relation．It divides a dataset into many groups and constructs one micro-cluster,which will be treated as a single point,corresponding to each group．All micro-clusters that are closed enough will be connected and put into the same group by the equivalence relation．The center of each group will be calculated and will be the center of a real cluster in the dataset．Experiments show that bigKClustering not only runs fast and obtains high clustering quality but also scales well.

Key words: Micro-cluster,Equivalence relation,Hadoop,Clustering