DEM Intercomparison eXercise (DEMIX) - Maps of completeness criteria scores for global DEMs

Introduction

This introduction gives a brief overview of the context in which the dataset has been produced. Readers curious about the detailed standards and procedures described in this section are encouraged to open the resources linked to this dataset.

The Digital Elevation Model Intercomparison eXercise (DEMIX)

This work is part of the Digital Elevation Model Intercomparison eXercise (DEMIX), initiated by the Committee on Earth Observation Satellites (CEOS). This initiative aims at "providing harmonised terminology and methods, as well as practical guidelines and results allowing the intercomparison of continental or global Digital Elevation Models (DEM)" (Strobl et al., 2021). Several publications have defined the framework of DEMIX, from the terminology and definitions (Guth et al., 2021) to the DEM ranking methods (Bielski et al., 2024). An additional methodology paper has been publicated regarding the assessment of planimetric displacements between DEMs (Riazanoff et al., 2024), which are a common source of biases in DEM comparisons.

The DEMIX grid

Studies performed within the DEMIX framework rely on the DEMIX grid (Guth et al., 2023), a geodetic grid (EPSG:4326) dividing the world in areas of approximately 10x10km. These standard areas are called DEMIX tiles, and can be precisely located thanks to their identifier.

Criteria and scores

Within DEMIX, several criteria have been defined to assess the quality of DEMs. These criteria take as input a DEM and a DEMIX tile, and provide as output the score of the DEM for this specific tile. Repeating this process over several DEMs and DEMIX tiles of interest allow for a comparison of scores, leading to a ranking of DEMs. DEMIX rankings are based on the Randomized Complete Block Design (RCBD) (Bielski et al., 2024).

This dataset

This dataset is composed of global maps of one map per (DEM, criterion) tuple. Each GeoTIFF map can be superimposed with the DEMIX grid (Guth et al., 2023) in a GIS (tested in QGIS 3.16).

Completeness criteria

The completeness criteria have originally been defined by Peter Strobl. A brief description of each criterion is given in the next table. Please see the column "Original document" and files of this repository for the complete definitions.

Criterion
Description
Requirements
 Original document

A01 - Product fractional cover
Fraction of a DEMIX tile covered by the DEM product
None
See document "DEMIX_CDD-A01_20211103.docx"

A02 - Valid data fraction
Fraction of a DEMIX tile covered by valid pixels of the DEM product
"No data" or "void" value in metadata
See document "DEMIX_CDD-A02_20211103.docx"

A03 - Primary data fraction
Fraction of a DEMIX tile covered by valid pixels generated from the main source of data of the DEM product
"No data" or "void" value in metadata + source data/editing mask
See document "DEMIX_CDD-A03_20211103.docx"

A04 - Valid land fraction
Fraction of a DEMIX tile covered by valid pixels of land of the DEM product
"No data" or "void" value in metadata + water body mask
See document "DEMIX_CDD-A04_20211103.docx"

A05 - Primary land fraction
Fraction of a DEMIX tile covered by valid pixels of land generated from the main source of data of the DEM product
"No data" or "void" value in metadata + water body mask + source data/editing mask
See document "DEMIX_CDD-A05_20211103.docx"

DEMs and ancillary data

The following DEM products and ancillary layers have been used to generate the dataset.

Identifier
Used layers
Data access

ASTGTM v003

ASTER GDEM elevations (dem.tif) + editing / source masks (num.tif)
https://lpdaac.usgs.gov/products/astgtmv003/

ASTWBD v001

ASTER GDEM water body mask (att.tif)
https://lpdaac.usgs.gov/products/astwbdv001/

AW3D30 v2003

ALOS World 3D elevations (DSM.tif) + editing / source / water body masks (MSK.tif)
https://www.eorc.jaxa.jp/ALOS/en/dataset/aw3d30/aw3d30_e.htm

COP-DEM_GLO-30-DGED v2019_1
Copernicus DEM GLO-30 elevations (DEM.tif) + editing (EDM.tif) + source (SRC.tif) + water body (WBM.tif) masks
https://spacedata.copernicus.eu/collections/copernicus-digital-elevation-model

COP-DEM_GLO-90-DGED v2019_1
Copernicus DEM GLO-90 elevations (DEM.tif) + editing (EDM.tif) + source (SRC.tif) + water body (WBM.tif) masks
https://spacedata.copernicus.eu/collections/copernicus-digital-elevation-model

NASADEM_HGT v001

NASADEM elevations (.hgt) + editing / source (.num) + water body (.swb) masks
https://lpdaac.usgs.gov/products/nasadem_hgtv001/

SRTMGL1 v003

SRTMGL1 elevations (.hgt)
https://lpdaac.usgs.gov/products/srtmgl1v003/

SRTMGL1N v003

SRTMGL1 editing / source / water body masks (.num)
https://lpdaac.usgs.gov/products/srtmgl1nv003/

Computation of scores

For each DEMIX tile and DEM, each "fractional cover" has been computed using the following procedure:

Crop DEMIX tile layers - The tiles of each DEM layer (elevations, editing, sources and water bodies) are cropped to the extent of the DEMIX tile.

Compute standardized layers - Given the cropped DEM layers, four standardized layers are produced, which are:

Heights layer - Containing the heights of the DEM

Land/water mask layer - Indicating whether DEM pixels are land or water:

0 = NO_DATA

1 = BACKGROUND

2 = INVALID

3 = WATER

4 = LAND

Source mask layer - Indicating the source data of DEM heights (or "edited" value):

0 = NO_DATA

1 = BACKGROUND

2 = INVALID

3 = PRIMARY_DATA

4 = EXTERNAL_DATA

5 = EDITED

Valid mask layer - Indicating if the DEM pixels are valid or not:

0 = NO_DATA

1 = BACKGROUND

2 = INVALID

3 = VALID

Retrieve pixel number N - The total pixel number N is computed for one of the layers (all layers have the same number of pixels).

Retrieve criterion pixel number C - The criterion pixel number C is computed based on the standard layers, more precisely:

A01 - Product fractional cover - Number of pixels of valid mask layer equal to 1, 2 or 3

A02 - Valid data fraction - Number of pixels of valid mask layer equal to 3

A03 - Primary data fraction - Number of pixels of source mask layer equal to 3

A04 - Valid land fraction - Number of pixels of land/water mask layer equal to 4

A05 - Primary land fraction - Number of pixels of source mask layer equal to 3 and land/water mask layer equal to 4

Compute the final score S - The final score S is expressed as the following percentage: S = ceil(C/N*100)

Known issues

The "SRTMGL1N v003" is known to have "tile repeating issues", where part of the data is wrongly flagged as water. This issue has been reported with no particular response from the providers of the DEM (see https://forum.earthdata.nasa.gov/viewtopic.php?t=2752).

References:

Guth, P.L.; Strobl, P.; Gross, K.; Riazanoff, S. DEMIX 10k Tile Data Set (1.0) [Data set]. Zenodo 2023. https://doi.org/10.5281/zenodo.7504791

Guth, P.L.; Van Niekerk, A.; Grohmann, C.H.; Muller, J.-P.; Hawker, L.; Florinsky, I.V.; Gesch, D.; Reuter, H.I.; Herrera-Cruz, V.; Riazanoff, S.; López-Vázquez, C.; Carabajal, C.C.; Albinet, C.; Strobl, P. Digital Elevation Models: Terminology and Definitions. Remote Sens. 2021, 13, 3581. https://doi.org/10.3390/rs13183581

Riazanoff, S.; Corseaux, A.; Albinet, C.; Strobl, P.A.; López-Vázquez, C.; Guth, P.L.; Tadono, T. Best BiCubic Method to Compute the Planimetric Misregistration between Images with Sub-Pixel Accuracy: Application to Digital Elevation Models. ISPRS Int. J. Geo-Inf. 2024, 13, 96. https://doi.org/10.3390/ijgi13030096

Bielski, C.; López-Vázquez, C.; Grohmann, C.H.; Guth, P.L.; Hawker, L.; Gesch, D.; Trevisani, S.; Herrera-Cruz, V.; Riazanoff, S.; Corseaux, A.; Reuter, H.I.; Strobl, P.A.; Novel Approach for Ranking DEMs: Copernicus DEM Improves One Arc Second Open Global Topography in IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-22, 2024, Art no. 4503922. https://doi.org/10.1109/TGRS.2024.3368015

Strobl, P.A.; Bielski, C.; Guth, P.L.; Grohmann, C.H.; Muller, J.P.; López-Vázquez, C.; Gesch, D.B.; Amatulli, G.; Riazanoff, S.; Carabajal, C. The Digital Elevation Model Intercomparison eXperiment DEMIX, a community based approach at global DEM benchmarking. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2021, XLIII-B4-2021, 395–400. https://doi.org/10.5194/isprs-archives-XLIII-B4-2021-395-2021