DGGS Benchmark Replication Study - Results Dataset

DGGS Benchmark Replication Study - Results Dataset Description This dataset contains the results of a reproducible replication study of the benchmarks presented in: Law, R.M. & Ardo, J. (2024). "Using a discrete global grid system for a scalable, interoperable, and reproducible system of land-use mapping." Big Earth Data, 9(1), 29-46. DOI: 10.1080/20964471.2024.2429847 The replication validates the paper's two central claims: Vector benchmark: DGGS provides "orders of magnitude" performance improvement over traditional vector overlay operations Raster benchmark: DGGS and raster methods show "roughly equivalent performance" for classification tasks Files Included File Description vector_benchmark.csv Timing results for vector overlay vs DGGS comparison raster_benchmark.csv Timing results for raster vs DGGS comparison indexing_benchmark.json Comparison of H3 loop vs xdggs vectorized indexing system_info.json Hardware/software environment details for reproducibility summary.json Structured summary of results and validation status benchmark_unified.png Visualization of all benchmark results (PNG format) benchmark_unified.pdf Visualization of all benchmark results (PDF format) Benchmark Configuration Vector Benchmark (Figure 6 replication) Layers tested: 5, 10, 20, 50 H3 resolution: 14 (matching paper) Method: Voronoi polygons with random point distribution, dissolved by binary value, then overlaid Raster Benchmark (Figure 7 replication) Layers tested: 10, 50, 100, 500, 1,000, 5,000, 10,000 H3 resolution: 9 Raster size: 100 × 100 pixels per layer Method: Neutral Landscape Model (mid-point displacement) with Gaussian smoothing Classification Logic Following the paper's methodology, classification uses seven mathematical functions applied to summed layer values: Prime number test Perfect number test Triangular number test Square number test Pentagonal number test Hexagonal number test Fibonacci number test The combination of these seven binary outputs produces up to 127 distinct classes. Key Results Vector Benchmark Layers DGGS Time (s) Vector Time (s) DGGS Speedup 5 ~0.02 ~0.4 22× 10 ~0.03 ~2.5 105× 20 ~0.05 ~27 541× 50 ~0.13 ~780 5,999× Conclusion: DGGS is orders of magnitude faster than vector overlay. The speedup increases with layer count because vector overlay creates exponentially more sliver polygons, while DGGS cell count remains fixed. Raster Benchmark DGGS pre-indexed vs Raster classification: Roughly equivalent performance (within 2-3×) xdggs vectorized indexing: Significantly faster than H3 loop-based indexing for coordinate-to-cell conversion Conclusion: For pre-indexed data, DGGS classification performance matches raster, validating the paper's claim. Understanding the Raster Benchmark Plot The raster benchmark plot shows four methods: Line Method What it measures 🟠 Raster (baseline) NumPy array operations Traditional raster stacking and classification 🟦 DGGS+H3 (reproduction) H3 loop indexing Paper's original approach: index each layer with H3, then classify 🟣 DGGS+xdggs (replication) xdggs vectorized indexing Alternative approach: index with xdggs, then classify 🟢 DGGS pre-indexed Read from Parquet Paper's target scenario: data already indexed to DGGS Key interpretation: The DGGS pre-indexed line (green) represents the paper's main use case: data is indexed to DGGS once, then queried many times The Classification Only subplot (bottom-right) isolates this comparison, showing DGGS and raster are roughly equivalent The gap between DGGS+H3 and DGGS+xdggs demonstrates the indexing speedup from vectorization Methodology What is a DGGS? A Discrete Global Grid System (DGGS) is a spatial reference system that partitions the Earth's surface into a hierarchical sequence of equal-area cells. Unlike traditional coordinate systems, DGGS provides: Fixed discretization: Space is divided into a finite number of cells at each resolution level Hierarchical structure: Cells nest within parent cells, enabling multi-resolution analysis Unique cell identifiers: Each cell has a unique ID that implicitly encodes its location This study uses H3, Uber's hexagonal hierarchical spatial index, which tessellates the globe with hexagonal cells (and 12 pentagons). H3 has 16 resolution levels, from ~4 million km² (resolution 0) to ~0.9 m² (resolution 15) per cell. Key insight: When data is indexed to a DGGS, spatial joins become simple attribute joins on cell IDs, avoiding expensive geometric intersection computations. Reproduction vs Replication Following established terminology in reproducibility research: Term Definition Implementation in this study Reproduction Same methodology, same tools H3 library + Polars (matching the paper's approach) Replication Same methodology, alternative tools xdggs for vectorized H3 indexing The Role of xdggs in Replication xdggs is a Python library that provides Xarray extensions for DGGS operations. It offers an alternative implementation for converting geographic coordinates to DGGS cell IDs. Why xdggs matters for this replication: The paper's workflow has two main phases: Indexing phase: Convert raster pixels or vector geometries to H3 cell IDs Classification phase: Query the indexed data to assign land-use classes The original paper used the H3 Python library, which requires looping through each coordinate to convert it to a cell ID: # H3 loop-based indexing (paper's approach) cell_ids = [h3.latlng_to_cell(lat, lng, resolution) for lat, lng in coordinates] # Slow: one call per pixel xdggs provides vectorized operations that convert all coordinates in a single call: # xdggs vectorized indexing (replication approach) cell_ids = h3_info.geographic2cell_ids(lngs, lats) # Fast: one call for all pixels Performance comparison (from this replication): Method Time per layer Relative speed H3 loop ~0.15s 1× (baseline) xdggs vectorized ~0.001s ~150× faster This demonstrates that while the paper's methodology is sound, alternative tools can significantly improve the indexing phase performance without changing the scientific conclusions. Implications for the Paper's Claims The replication with xdggs validates the paper's conclusions: Vector benchmark: The speedup comes from replacing geometric intersection with cell ID joins—this is independent of the indexing method used Raster benchmark: Pre-indexed DGGS data performs equivalently to raster regardless of how the indexing was done The xdggs results show that the DGGS approach can be even more efficient with optimized tooling, strengthening the paper's argument for DGGS adoption. Software Environment Python 3.11 H3 v4.x (Uber's hexagonal hierarchical spatial index) xdggs (vectorized DGGS operations) GeoPandas, Rasterio, NumPy, Pandas, Polars SciPy (Voronoi tessellation, Gaussian filtering) How to Reproduce The complete replication environment is available at: GitHub: https://github.com/annefou/dggs-benchmark-replication Using Docker (Recommended) docker pull ghcr.io/annefou/dggs_replication_2026:latest docker run -v $(pwd)/results:/app/results ghcr.io/annefou/dggs_replication_2026:latest Using Python git clone https://github.com/annefou/dggs-benchmark-replication.git cd dggs-benchmark-replication pip install -r requirements.txt python run_replication.py --all Citation If you use this dataset, please cite both the original paper and this replication: Original Paper @article{law2024dggs, title={Using a discrete global grid system for a scalable, interoperable, and reproducible system of land-use mapping}, author={Law, Richard M. and Ardo, James}, journal={Big Earth Data}, volume={9}, number={1}, pages={29--46}, year={2024}, publisher={Taylor \& Francis}, doi={10.1080/20964471.2024.2429847} } This Replication Dataset @dataset{fouilloux2026dggs_replication, author = {Fouilloux, Anne}, title = {{DGGS Benchmark Replication Study: Results Dataset}}, year = {2026}, publisher = {Zenodo}, doi = {10.5281/zenodo.XXXXXXX}, url = {https://doi.org/10.5281/zenodo.XXXXXXX} } Original Benchmark Code The original benchmark code from the paper is available at: Repository: https://github.com/manaakiwhenua/dggsBenchmarks Version used: v1.1.1 License This dataset is released under the MIT License. Author Anne FouillouxORCID: 0000-0002-1784-2920Affiliation: LifeWatch ERIC Acknowledgments Richard M. Law and James Ardo (Manaaki Whenua – Landcare Research) for the original research Uber Technologies for the H3 library The xdggs development team for vectorized DGGS operations Related Resources DGGS and H3 H3 Documentation: https://h3geo.org/ H3 Python API: https://uber.github.io/h3-py/ H3 Resolution Table: https://h3geo.org/docs/core-library/restable/ OGC DGGS Standard: https://www.ogc.org/standard/dggs/ xdggs and Related Tools xdggs Documentation: https://xdggs.readthedocs.io/ xdggs GitHub: https://github.com/xarray-contrib/xdggs h3ronpy (used by xdggs): https://github.com/nmandery/h3ronpy Original Research Original paper: https://doi.org/10.1080/20964471.2024.2429847 Original benchmark code: https://github.com/manaakiwhenua/dggsBenchmarks vector2dggs tool: https://github.com/manaakiwhenua/vector2dggs raster2dggs tool: https://github.com/manaakiwhenua/raster2dggs Reproducibility Resources FORRT Replication Handbook: https://forrt.org/replication_handbook/ The Turing Way - Reproducibility: https://the-turing-way.netlify.app/reproducible-research/ Dataset generated: January 2026Replication framework version: 2.0.0