计算机科学 ›› 2016, Vol. 43 ›› Issue (3): 18-22.doi: 10.11896/j.issn.1002-137X.2016.03.003

• 目次 • 上一篇    下一篇

阵列众核结构上的一种多层分区Hash连接算法

石嵩,宁永波,李宏亮,郑方   

  1. 江南计算技术研究所 无锡214083,江南计算技术研究所 无锡214083,江南计算技术研究所 无锡214083,江南计算技术研究所 无锡214083
  • 出版日期:2018-12-01 发布日期:2018-12-01

Multi-layer Partition Hash Join Algorithm on Array-based Manycore Architecture

SHI Song, NING Yong-bo, LI Hong-liang and ZHENG Fang   

  • Online:2018-12-01 Published:2018-12-01

摘要: 连接是数据查询处理中最耗时、使用最频繁的操作之一,对提高连接操作的速率具有重要意义。阵列众核处理器是一类重要的众核处理器,具有强大的并行能力,可用来加速并行计算。基于阵列众核处理器的结构,设计和优化了一种高效的多层分区Hash连接算法。该算法通过多层划分的策略大大降低了主存访问次数,通过分区重排方法有效消除了数据倾斜的影响,获得了很高的性能。在异构融合阵列众核处理器DFMC(Deeply-Fused Many Core)原型系统上的实验结果表明,DFMC上多层分区Hash连接算法的性能是CPU-GPU耦合结构上最快的连接算法的8.0倍,表明利用阵列众核处理器加速数据查询应用具有优势。

关键词: 阵列众核,Hash连接,数据倾斜,并行算法

Abstract: Join is one of the most time-consuming and most frequently used operations in database query processing,so it has great importance to improve the speed of join operation.The array-based manycore processor is one of the most important classes of manycore processors and has great parallelism,and can be used to accelerate parallel computing.In this paper,we designed and optimized an efficient algorithm called multi-layer partition hash join algorithm on array-based manycore architecture.Our algorithm achieves high performance by multi-layer partitioning which reduces the times of main memory accessing,and by partition rearrangement which eliminates the influence of data skew.Experimental results show that the performance of multi-layer partition hash join algorithm on DFMC(Deeply-Fused Many Core) processors is 8.0x times higher than that of the fastest hash join on CPU-GPU coupled-architecture,which de-monstrates that array-based manycore processors have advantages in data query processing.

Key words: Array-based manycore,Hash join,Data skew,Parallel algorithm

[1] Ramey C.Tile-gx100 manycore processor:Acceleration inter-faces and architecture[R].Tilera Corporation,2011
[2] Sato M.Feasibility study on future HPC infrastructure[R].2014
[3] Gwennap L.Adapteva:More flops,less watts[J].Microprocessor Report,2011,6(13):11-12
[4] Dinechin B I T D,Massas P G D,Lager G,et al.A Distributed Run-Time Environment for the Kalray MPPA-256 Integrated Manycore Processor[J].Procedia Computer Science,2013,18:1654-1663
[5] Fan D,Yuan N,Zhang J,et al.Godson-T:An efficient many-core architecture for parallel program executions[J].Journal of Computer Science and Technology,2009,24(6):1061-1073
[6] Zheng Fang,Zhang Kun,Wu Gui-ming,et al.Architecture techniques of many-core processor for energy-efficient in high performance computing[J].Chinese Journal of Computers,2014,37(10):2176-2186(in Chinese) 郑方,张昆,邬贵明,等.面向高性能计算的众核处理器结构级高能效技术[J].计算机学报,2014,37(10):2176-2186
[7] He B,Lu M,Yang K,et al.Relational query coprocessing on graphics processors[J].ACM Transactions on Database Systems (TODS),2009,34(4):21
[8] Mostak T.An Overview of MapD (Massively Parallel Database)[R].Massachusetts Institute of Technology,2013
[9] Scherger M.Design of an in-memory database engine using Intel Xeon Phi coprocessors[C]∥PDPTA’14.2014
[10] He B,Yang K,Fang R,et al.Relational joins on graphics processors[C]∥Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.ACM,2008:511-524
[11] Kaldewey T,Lohman G,Mueller R,et al.GPU join processing revisited[C]∥Proceedings of the Eighth International Workshop on Data Management on New Hardware.ACM,2012:55-62
[12] He J,Lu M,He B.Revisiting co-processing for hash joins on the coupled cpu-gpu architecture[J].Proceedings of the VLDB Endowment,2013,6(10):889-900
[13] Petrides P,Diavastos A,Christofi C,et al.Scalability and Efficiency of Database Queries on Future Many-Core Systems[C]∥2013 21st Euromicro International Conference on Parallel,Distributed and Network-Based Processing (PDP).IEEE,2013:24-28
[14] Kim C,Sedlar E,Chhugani J,et al.Sort vs.hash revisited:Fast join implementation on modern multi-core CPUs[J].PVLDB,2009,2(2):1378-1389
[15] Gray J,Sundaresan P,Englert S,et al.Quickly Generating Billion-Record Synthetic Databases[J].ACM Sigmod Record,1994,3(2):243-253

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!