计算机科学 ›› 2022, Vol. 49 ›› Issue (10): 66-73.doi: 10.11896/jsjkx.220100089

• 高性能计算* 上一篇    下一篇

基于新一代神威超算的计算流体力学Palabos软件的并行优化

柳安军1, 殷洪辉2, 王利1, 刘智翔3, 孔博4, 郭猛1,2, 陈成敏2, 杨美红1   

  1. 1 齐鲁工业大学(山东省科学院)山东省计算中心(国家超级计算济南中心) 济南 250014
    2 济南超级计算技术研究院济南市高性能工业软件重点实验室 济南 251013
    3 上海海洋大学信息学院 上海 201306
    4 广东以色列理工学院 广东 汕头 526100
  • 收稿日期:2022-01-10 修回日期:2022-05-09 出版日期:2022-10-15 发布日期:2022-10-13
  • 通讯作者: 杨美红(yangmh@sdas.org)
  • 作者简介:(liuaj@sdas.org)
  • 基金资助:
    国家重点研发计划(2018YFB0704002);鳌山科技创新计划重大项目(2018ASKJ01);山东省重大科技创新工程项目(2019JZZY010302);山东省重点研发计划(国际科技合作)(2019GHZ018);山东省博士后人才创新支持计划(SDBX2020018);光合基金B (202107021062)

Parallel Optimization of Computational Fluid Dynamics Application Palabos Based on NextGeneration Sunway Supercomputer

LIU An-jun1, YIN Hong-hui2, WANG Li1, LIU Zhi-xiang3, KONG Bo4, GUO Meng1,2, CHEN Cheng-min2, YANG Mei-hong1   

  1. 1 Shandong Computer Science Center(National Supercomputing Center in Jinan),Qilu University of Technology(Shandong Academy of Sciences),Jinan 250014,China
    2 Jinan Key Laboratory of High Performance Industrial Software,Jinan Institute of Supercomputer Technology,Jinan 251013,China
    3 College of Information Technology,Shanghai Ocean University,Shanghai 201306,China
    4 Guangdong Technion Israel Institute of Technology,Shantou,Guangdong 526100,China
  • Received:2022-01-10 Revised:2022-05-09 Online:2022-10-15 Published:2022-10-13
  • About author:LIU An-jun,born in 1990,Ph.D.His main research interests include parallel computing and mass/momentum/heat transfer.
    YANG Mei-hong,born in 1966,postgraduate,professor,Ph.D,supervisor.Her main research interests include cloud computing,big data and software engineering.
  • Supported by:
    National Key Research and Development Program(2018YFB0704002),Aoshan Science and Technology Innovation Project(2018ASKJ01), Major Scientific and Technological Innovation Projects in Shandong Province(2019JZZY010302),Shandong Key Research and Development Program(International Cooperation Office)(2019GHZ018),Shandong Province Postdoctoral Innovative Talents Support Plan(SDBX2020018) and GH fund B(202107021062).

摘要: Palabos软件是一款基于格子玻尔兹曼算法(Lattice Boltzmann Method,LBM)的计算流体力学软件,因其优异的计算能力被广泛用于多孔介质、自由界面、颗粒运动、血液流动等计算流体力学领域。Palabos软件广泛的用户需求使其迫切需要在神威超算上进行移植优化和并行加速,服务于能源、化工行业。文中在新一代神威超算(SW26010pro)上对Palabos软件进行异构并行设计,针对Palabos的数据结构和模块化编程不利于神威众核编程的问题,通过直接取址,设置字段标记处理多态导致的分支、数据切片处理等优化思路;并结合新一代神威超算的特性,使用共享内存和寄存器通信的优化技术,实现众核加速2~6倍。同时实现Palabos软件在新一代神威超算上的复杂化工过程多尺度计算方向上两相流算法的百万核心规模的并行计算,以6.4万核心的并行计算规模为基准,百万核心的并行效率大于40%。

关键词: 众核化, 模块化编程, Palabos, 新一代神威超算, 多相流

Abstract: Parallel lattice Boltzmann(Palabos)software is a widely used computational fluid dynamics software based on lattice Boltzmann method(LBM),which is widely used in the field of porous media,free interface,particle motion,blood flow and so on due to its excellent computing power.Palabos has a wide range of user needs,which makes it urgent to transplant,optimize and accelerate parallel on Sunway supercomputer to serve the energy and chemical industry.In this paper,the heterogeneous parallel design of Palabos software is carried out on the new generation Sunway supercomputer system(SW26010pro).The data structure and template programming of Palabos are not suitable for the heterogeneous parallel of Sunway supercomputer system.So we design the parallel optimization techniques called direct getting address,polymorphic tag processing and data slicing to deal with the Palabos data structure and template programming.Combined with the characteristics of the new generation of Sunway supercomputer system,the optimization technology of shared memory and register memory access(RMA) is also adopted.The acceleration efficiency of 64 computing processing elements(CPEs) is 2~6 speed up.The Palabos software is realized the parallel computing of one million core scale of two-phase flow algorithm in the field of complex multi-scale chemical process in the new generation Sunway supercomputer system.The one million cores parallel efficiency is more than 40% compared with 64 000 cores.

Key words: Many core, Modulation programming, Palabos, SW26010pro, Multiphase flow

中图分类号: 

  • TP391
[1]LV X J,LIU Z,CHU X S,et al.Extreme-scale simulation based LBM computing fluid dynamics simulations[J].Computer Science,2020,47(4):13-17.
[2]LIU Z,CHU X S,LV X J,et al.SunWayLB:Enabling extreme-scale Lattice Boltzmann Method based computing fluid dynamics simulations on Sunway TaihuLight[C]//2019 IEEE International Parallel and Distributed Processing Symposium(IPDPS).IEEE Computer Society,2019:557-566.
[3]LIU Z X,FANG Y,SONG A P,et al.Large-Scale scalable parallel computing based on LBM with multiple-relaxation-time model[J].Journal of Computer Research and Development,2016,53(5):1156-1165.
[4]TIAN M,GU W,PAN J,et al.In Performance analysis and optimization of palabos on petascale sunway BlueLight MPP supercomputer[C]//International Conference on Parallel Computing in Fluid Dynamics.Springer,2013:311-320.
[5]OBRECHT C,KUZNIK F,TOURANCHEAU B,et al.TheTheLMA project:a thermal lattice Boltzmann solver for the GPU [J/OL].Computers & Fluids,2014,54:118-126.https://doi.org/10.1016/j.compfluid.2011.10.011.
[6]YE H,SHEN Z,XIAN W,et al.OpenFSI:A highly efficient and portable fluid-structure simulation package based on immersed-boundary method[J/OL].Computer Physics Communications,2020:107463.https://doi.org/10.1016/j.cpc.2020.107463.
[7]BONACCORSO F,MONTESSORI A,TIRIBOCCHI A,et al.LBsoft:a parallel open-source software for simulation of colloidal systems[J/OL].Computer Physics Communications,2020:107455.https://doi.org/10.1016/j.procs.2017.05.084.
[8]ZAVODSZKY G,VAN ROOIJ B,AZIZI V,et al.Hemocell:ahigh-performance microscopic cellular library[J/OL].Procedia Computer Science,2017,108:159-165.https://doi.org/10.1016/j.procs.2017.05.084.
[9]HASERT M,MASILAMANI K,ZIMNY S,et al.Complex fluid simulations with the parallel tree-based lattice Boltzmann solver Musubi[J].Journal of Computational Science,2014,5 (5):784-794.
[10]LATT J,MALASPINAS O,KONTAXAKIS D,et al.Palabos:Parallel Lattice Boltzmann Solver[J].Computers & Mathema-tics with Applications,2021,81(1):334-350.
[11]LATT J,CHOPARD B.VLADYMIR—a C++ matrix library for data-parallel applications[J].Future Generation Computer Systems,2004,20(6):1023-1039.
[12]MOHAMMADREZAEI S,SIAVASHI M,ASIAEI S.Surfacetopography effects on dynamic behavior of water droplet over a micro-structured surface using an improved-VOF based lattice Boltzmann method[J/OL].Journal of Molecular Liquids,2022:118509.https://doi.org/10.1016/j.molliq.2022.118509.
[13]XIA T,FENG Q,WANG S,et al.A numerical study of particle migration in porous media during produced water reinjection[J/OL].Journal of Energy Resources Technology,2022,144 (7):073002.https://doi.org/10.1115/1.4052165.
[14]ZAVODSZKY G,VAN ROOIJ B,CZAJA B,et al.Red bloodcell and platelet diffusivity and margination in the presence of cross-stream gradients in blood flows[J/OL].Physics of Fluids,2019,31(3):031093.https://doi.org/10.1063/1.5085881.
[15]KOTSALOS C,LATT J,CHOPARD B.Bridging the computational gap between mesoscopic and continuum modeling of red blood cells for fully resolved blood flow[J/OL].Journal of Computational Physics,2019,398:108905.https://doi.org/10.1016/j.jcp.2019.108905.
[16]KOTSALOS C,LATT J,BENY J,et al.Digital blood in mas-sively parallel CPU/GPU systems for the study of platelet transport[J/OL].Interface Focus:a Theme Supplement of Journal of the Royal Society Interface,2021,11(1):20190116.https://doi.org/10.1098/rsfs.2019.0116.
[17]BOUDJELTIA K Z,KOTSALOS C,DRIBEIRO D,et al.Spherization of red blood cells and platelet margination in COPD patients[J].Annals of the New York Academy of Sciences,2021,1485(1):71-82.
[18]LIU Y,LIU X,LI F,et al.In Closing the “quantum supremacy” gap:achieving real-time simulation of a random quantum circuit using a new Sunway supercomputer[C/OL]//Proceedings of the International Conference for High Performance Computing,Networking,Storage and Analysis.IEEE Computer Society,2021.https://doi.org/10.48550/arXiv.2110.14502.
[19]XIAO J,CHEN J,ZHENG J,et al.In Symplectic structure-preserving particle-in-cell whole-volume simulation of tokamak plasmas to 111.3 trillion particles and 25.7 billion grids[C/OL]//Proceedings of the International Conference for High Perfor-mance Computing,Networking,Storage and Analysis.IEEE Computer Society,2021.https://doi.org/10.1145/3458817.3487398.
[20]SHANG H,LI F,ZHANG Y,et al.In Extreme-scale ab initio quantum raman spectra simulations on the leadership HPC system in China[C/OL]//Proceedings of the International Confe-rence for High Performance Computing,Networking,Storage and Analysis.IEEE Computer Society,2021.https://doi.org/10.1145/3458817.3487402.
[1] 徐传福,王曦,刘舒,陈世钊,林玉.
基于Python的大规模高性能LBM多相流模拟
Large-scale High-performance Lattice Boltzmann Multi-phase Flow Simulations Based on Python
计算机科学, 2020, 47(1): 17-23. https://doi.org/10.11896/jsjkx.190500009
[2] 段兴锋,任鸿翔.
基于物理的海浪模拟综述
Overview of Physically-based Ocean Waves Simulation
计算机科学, 2014, 41(8): 1-6. https://doi.org/10.11896/j.issn.1002-137X.2014.08.001
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!