计算机科学 ›› 2017, Vol. 44 ›› Issue (3): 20-22.doi: 10.11896/j.issn.1002-137X.2017.03.005

• 2015全国高性能计算学术年会 • 上一篇    下一篇

异构集群上的宏基因组聚类优化

韦建文,许志耿,王丙强,Simon SEE,林新华   

  1. 上海交通大学高性能计算中心 上海200240,上海交通大学高性能计算中心 上海200240,国家超算深圳中心 深圳518055,上海交通大学高性能计算中心 上海200240;NVIDIA公司 新加坡,上海交通大学高性能计算中心 上海200240;NVIDIA公司 新加坡
  • 出版日期:2018-11-13 发布日期:2018-11-13
  • 基金资助:
    本文受国家高技术研究发展计划(863):高性能计算环境应用服务优化关键技术研究(2014AA01A302),日本学术振兴会(RONPAKU Fellowship)资助

Accelerating Gene Clustering on Heterogeneous Clusters

WEI Jian-wen, XU Zhi-geng, WANG Bing-qiang, Simon SEE and James LIN   

  • Online:2018-11-13 Published:2018-11-13

摘要: 宏基因组基因聚类是筛选致病基因的新型方法,其依赖于海量的测序数据、有效的聚类算法以及高效的计算机来实现。相关系数矩阵的计算是进行聚类前必须完成的操作,占总计算量的比重较大。以某基因库为例,包含1300个样本、每样本百万基因的数据,单线程运行需要27年。充分发挥多核CPU的潜力,利用GPU加速卡强大的计算能力,将程序扩展到多节点集群上运行,是重要而迫切的工作。在仔细分析算法的基础上,首先针对单CPU节点和单GPU卡做了高效实现,获得了接近理想的加速比;然后利用缓存优化进一步提升性能;最后使用负载均衡方法在MPI线程间分发计算任务,实现了良好的扩展。相比未优化的单线程程序,16节点CPU获得了238.8倍的加速,6 块GPU卡获得了263.8倍的加速。

关键词: 基因聚类,异构计算,缓存优化,负载均衡

Abstract: Metagenome clustering is a novel approach to detect flaw genes which relies on massive gene data,effective clustering algorithms and efficient implementation.In clustering,calculating correlation matrix is essential,accounting most of computing time.To take a gene repo as an example,which has 1300 samples and million genes,it will take about 27 years to cluster them.Therefore,developing efficient implementations for calculating correlation matrix is most essential.After analyzing the algorithms,we proposed and took several optimization approaches.First,we implemented an efficient multithread one using OpenMP dynamic scheduling.Secondly,we further improved the performance by utilizing cache on CPU and shared memory on GPU efficiently.Thirdly,we implemented a loadbalance work distribution which works well on the MPI program on CPU.Compared to the unoptimized single-threaded CPU program,the two fasted one,MPI+OpenMP on 256 CPU cores and MPI+CUDA on 6 GPU cards,achieve 238.8 and 263.8 speedups.

Key words: Gene clustering,Heterogeneous computing,Cache optimization,Load balance

[1] QIN J,LI Y,CAI Z,et al.A metagenome-wide association study of gut microbiota in type 2 diabetes[J].Nature,2012,490(7418):55-60.
[2] GUO G X,LU X J,QIU S,et al.GPU-accelerated Gene Clustering Method for Metagenome [C]∥Proceedings of HPC China 2014.Guangzhou,2014:324-328.(in Chinese)郭贵鑫,陆旭佳,邱爽,等.基于GPU加速的宏基因组聚类方法[C]∥HPC China 2014会议.广州,2014:324-328.
[3] WANG M,ZHANG W,DING W,et al.Parallel Clustering Algorithm for Large-Scale Biological Data Sets[J].PLoS ONE,2014,9(4):13-15.
[4] W W M,KIRK D B.Programming Massively Parallel Processors [M].New York:Morgan Kaufmann,2012:120-128.
[5] EKANAYAKE V,GUNARATHNE T,QIU J.Cloud Techno-logies for Bioinformatics Applications[J].IEEE Transactions on Parallel and Distributed Systems,2011,22(6):998-1011.
[6] BUSTAMAM A,BURRAGE K,HAMILTON N A.Fast Parallel Markov Clustering in Bioinformatics Using Massively Parallel Computing on GPU with CUDA and ELLPACK-R Sparse Format[J].IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 2012,9(3):234-240.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!