计算机科学 ›› 2012, Vol. 39 ›› Issue (1): 134-137.

• 数据库与数据挖掘 • 上一篇    下一篇

面向大规模数据的快速并行聚类划分算法研究

牛新征 佘堑   

  1. (电子科技大学计算机科学与工程学院 成都610054)
  • 出版日期:2018-11-16 发布日期:2018-11-16

Study of Fast Parallel Clustering Partition Algorithm for Large Data Sets

  • Online:2018-11-16 Published:2018-11-16

摘要: 随着聚类分析中处理数据量的急剧增加,面对大规模数据,传统K-Mcans聚类算法面临着巨大挑战。为了提高传统K-Means聚类算法的效率,针对已有基于MPI的并行K-Means聚类算法和基于Hadoop的分布式K-Means云聚类算法,从聚心初始化和通信模式等入手,提出了改进思路和具体实现。实验结果表明,所提算法能大大减少通信量和计算量,具有较高的执行效率。研究结果可以为以后设计更好的大规模数据快速并行聚类划分算法提供研究依据。

关键词: 云计算,K-Means,大规模数据,MPI, Hadoop

Abstract: With the rapid increase of data amounts in clustering algorithms' processing, traditional K-Means clustering algorithm is facing huge challenge for large data sets. In order to improve efficiency of traditional K-Means clustering algorithm, this paper proposed some improvement ideas and implementation using the cluster center initialization and communication mode, according to parallel clustering algorithm based on MPI and distributed clustering algorithm based on Hadoop in cloud. The results show that research of the algorithm can reduce the communication and computation largely, and can have higher implementation efficiency. I}hc research fruits will help us to design better and fast parallel clustering partition algorithm for large data sets in future.

Key words: Cloud computing, K-means, Large data sets, Message passing interface, Hadoop

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!