Geofluid Object Workbench (GeoFLOW) for Atmospheric Dynamics in the Approach to Exascale: Spectral Element Formulation and CPU Performance Monthly Weather Review

A new software framework using a well-established high-order spectral element discretization is presented for solving the compressible Navier–Stokes equations for purposes of research in atmospheric dynamics in bounded and unbounded limited-area domains, with a view toward capturing spatiotemporal intermittency that may be particularly challenging to attain using low-order schemes. A review of the discretization is provided, emphasizing properties such as the matrix product formalism and other design considerations that will facilitate its effective use on emerging exascale platforms, and a new geometry-independent, element boundary exchange method is described to maintain continuity. A variety of test problems are presented that demonstrate accuracy of the implementation primarily in wave-dominated or transitional flow regimes; conservation properties are also demonstrated. A strong scaling CPU study in a three-dimensional domain without using threading shows an average parallel efficiency of ≳99% up to 2 × 104 MPI tasks that is not affected negatively by expansion polynomial order. On-node performance is also examined and reveals that, while the primary numerical operations achieve their theoretical arithmetic intensity, the application performance is largely limited by available memory bandwidth. Significance Statement This work considers the need for computationally efficient, high-order, low dissipation numerics to fully leverage emerging exascale computing resources in an effort to examine and improve the accuracy of numerical treatments of atmospheric and weather phenomena. A new spectral element implementation is introduced that attempts to address the issues involved. Well-understood tests are presented that illustrate the known efficacy of the method in wave-dominated, quasi-laminar, and relatively strong shear flow regimes, and good conservation properties for mass and total energy are achieved. Importantly, the implementation is shown to exhibit encouraging performance characteristics. Grant no. NA19OAR4320073