Coupling ISSM and CUAS-MPI: example cases

Coupling of ISSM and CUAS-MPI with preCICE: Sample Cases

This data set contains samples cases for coupling of the Ice-sheet and Sea-level System Model (ISSM) with subglacial hydrology model CUAS-MPI, an MPI-parallel implementation of the Confined-Unconfined Aquifer System model. The coupling is performed by the coupling library preCICE. ISSM computes the state of the ice sheet (e.g., ice thickness, ice velocity, melting rates) and CUAS-MPI computes the effective water pressure that is included in the sliding law used by ISSM.

Thule: A synthetic ice sheet, based on the Thule geometry developed for the CalvingMIP (https://github.com/JRowanJordan/CalvingMIP/wiki/) project. The sample can be run either fully coupling or one directional coupling. The sample is a simple demonstration of the capabilities of the adapters and demonstrates the effects of the feedback. The algorithmically generated geometry is just complex enough to see effects of coupling in areas of grounded and floating ice and at the margins between these areas. The setups were generated by simulating uncoupled in ISSM with the built-in effective pressure until the ice reaches steady state, then spinning up (also without coupling) a consistent hydrology based on the steady state geometry.

Greenland: A medium resolution model of the Greenland Ice Sheet based on setups G1000 from Fischler et al. (2022, doi 10.5194/gmd-15-3753-2022) and G600 from Fischler et al. (2023, doi 10.5194/gmd-16-5305-2023). These setups are not entirely consistent, so the simulation result is not realistic. The sample is mostly intended to produce representative performance measurements, enabling comparison of parallel and serial coupling (see below.)

Requirements

To run the cases, the following software is required (does not include transitive dependencies):- ISSM 4.24 (website)- preCICE 3.0 or higher (website)- ISSM-preCICE adapter 0.4 (gitlab, zenodo)- CUAS-MPI 0.1 with preCICE adapter (zenodo)

Installation instructions can be found in the referenced websites and source code archives.

Run Coupled Simulations

Sample cases include scripts to run the setups on a high-performance computing cluster. The scripts are specific for the Albedo cluster hosted by the Alfred Wegener Institute Helmholtz Center for Polar and Marine Research, but should run on other clusters with minor adaptations, including but not limited to:- Filesystem paths in queue_job.sh- Environment setup for participants in env.sh files (e.g. load modules)- Slurm directives for accounts, partitions, etc. in `queue_job.sh`- Replace srun with mpirun or any other equivalent command required by the cluster

To run the coupled setups, there is a queue_job.sh script in both greenland/coupled, thule/1way, and thule/2way directories. First edit the configuration in queue_job.sh (parallel or serial coupling, directories, slurm settings, etc)Then execute the script to queue a SLURM job.

Spin-ups

Both setups also contain spin-up directories to create the coupled setup data files from scratch.

Greenland 

Both spin-ups are run independently.

- ISSM spin-up: provided without data files, see Fischler et al. (2022, doi 10.5194/gmd-15-3753-2022) for the full setup and compare to see changes made for coupling.- CUAS spin-up: not included, see Fischler et al. (2023, doi 10.5194/gmd-16-5305-2023)

Thule

Process:1. run uncoupled ISSM spin-up2. export data files for CUAS spin-up: ISSM restart, initial CUAS input3. run CUAS spin-up coupled to ISSM4. export data files for coupling: ISSM and CUAS restart, seasonal forcing

Postprocessing 

Greenland

This case is used for analyzing computational performance. Experiment directory names follow the schema work-<coupling scheme>-<ISSM CPUs>-<CUAS CPUs>-<date>-<time>. For the profiling runs, builds of ISSM and CUAS-MPI were used that don't write any output, since variance of IO on the cluster is too high (modify the solver/adapter configs to produce output for diagnostics or other). Otherwise, the shell script were used as explained above. Profiling is enabled in the included config files.

The provided scripts were used to analyze the raw and aggregated profile data:- analyze.sh: use `precice-profiling` tools included in the preCICE distribution to extract profiles and traces from the raw data.- aggregate.py: load traces of on or more runs and compute statistics (e.g., averages over runs, tasks, and coupling windows).- plot_*.py: plot different aspects of performance results (scaling, mapping comparisons)- archive.sh: compress experiment directories into an archive.

Thule

This case is used to demonstrate functionality. Postprocessing steps:

- post-processing.sh: generate derived variables from output of 1way and 2way coupling simulations    - 1way coupling files do not contain the correct basal_velocity_ice fields, because they are not coupled    - same units dissagree (fix in post-processing)    - spatial averaging    - (optional) check results using CDO- plot_*: plot map and time series of results