Weak scaling of parallel FMM vs. FFT up to 4096 processes

This figure shows the weak scaling of a parallel FMM-based fluid solver on GPUs, from 1 to 4096 processes. The FMM (fast multipole method) is used as the numerical engine in a vortex method fluid solver, simulating decaying isotropic turbulence. The reference method for this application is the pseudo-spectral method, which uses FFT as the numerical engine. Given the communication pattern of FFT, only 14% parallel efficiency is obtained with the spectral method on 4096 processes (no GPU acceleration). The parallel efficiency of the FMM-based solver is 74% at 4096 processes (one GPU per MPI process, 3 GPUs per node). It is important to note that the results correspond to the full-application codes, not just the FMM and FFT algorithms. The spectral method calculations were done using the 'hit3d' code (see link below). The size of the largest problem corresponds to a 4096^3 mesh, i.e., almost 69 billion points (about 17 million points per process). These calculations were run on the TSUBAME 2.0 system of the Tokyo Institute of Technology, thanks to guest access, during Fall 2011. The figure is here shared under CC-BY. Please use the handle and doi above for citation if you use it.

本图展示了基于快速多极子方法（FMM, Fast Multipole Method）的并行GPU流体求解器在1至4096个进程下的弱可扩展性表现。该求解器以FMM作为涡旋法流体求解的数值引擎，用于模拟衰减各向同性湍流。本应用的参考方法为伪谱方法，其数值引擎为快速傅里叶变换（FFT, Fast Fourier Transform）。鉴于FFT的通信模式特征，该谱方法在4096个进程下仅能实现14%的并行效率（未启用GPU加速）。而基于FMM的流体求解器在4096个进程下的并行效率可达74%（每个MPI进程对应一块GPU，每个节点配置3块GPU）。需特别说明的是，本次实验结果对应完整的应用程序代码，而非仅针对FMM与FFT算法本身。伪谱方法的计算通过'hit3d'代码完成（详见下方链接）。本次实验的最大问题规模对应4096³的网格，即近690亿个计算点（每个进程约承担1700万个计算点）。上述计算于2011年秋季在东京工业大学的TSUBAME 2.0超级计算机系统上完成，实验得益于校方提供的访客访问权限。本图片采用知识共享署名（CC-BY）许可协议进行共享。若您使用本图片，请引用上方提供的标识符与DOI编号。