Integrated Approach to Global Land Use and Land Cover Reference Data Harmonization

INTRODUCTION This document details the process of creating a global inventory of reference samples and Earth Observation (EO) / gridded datasets for the Global Pasture Watch initiative. This inventory is used for training and validating machine-learning models for the GPW grassland mapping. This documentation outlines methodology, data sources, workflow, and results. Keywords: Grassland, Land Use, Land Cover, Gridded Datasets, Harmonization. OBJECTIVES Create a global inventory of existing reference samples for land use and land cover (LULC). Compile global EO / gridded datasets that capture LULC classes for a point or region and harmonize them to match the Global Pasture Watch classes. Develop automated scripts for data harmonization and integration. REPOSITORY STRUCTURE data/: Contains raw datasets and its metadata. workflow/: Methodology and reference ontology used for harmonizing datasets scripts/: Includes Python codes used for downloading, processing, and harmonizing datasets, and data analysis/visualization. results/: Final harmonized datasets in Parquet format. DATA COLLECTION The following datasets were incorporated : Land Change Monitoring, Assessment, and Projection (LCMap) (U.S.) MapBiomas samples (Brazil) Land Use/Land Cover Area Frame Survey LUCAS samples (Europe) GeoWiki G-GLOPS training dataset (Global) PREDICTS database samples (Global) WorldCereal samples (Global) DynamicWorld samples (Global) CropHarvest samples (Global) EuroCrops samples (Global) Global Land Cover Mapping and Estimation (GLanCE) samples (Global) Datasets Spatial distribution Time period Number of individual samples WorldCereal Global 2016-2021 38,267,911 GLanCE Global 1985-2021 31,061,694 EuroCrops Europe 2015-2022 14,742,648 GeoWiki G-GLOPS training dataset Global 2021 11,394,623 MapBiomas Brazil Brazil 1985-2018 3,234,370 Land Use/Land CoverArea Frame Survey (LUCAS) Europe 2006-2018 1,351,293 Dynamic World Global 2019-2020 1,249,983 Land Change Monitoring,Assessment, and Projection (LCMap) U.S. (CONUS) 1984-2018 874,836 GeoWiki 2012 Global 2011-2012 151,942 PREDICTS Global 1984-2013 16,627 Crop Harvest Global 2018-2021 9,714 Data files and metadata https://drive.google.com/drive/folders/1JNncgbxoW3-OeU-QCkRNnPGC6T-0efe_?usp=sharing WORKFLOW Harmonization Process This section details harmonizing global reference samples and EO/gridded datasets to align them with GPW classes, optimizing their integration into the GPW machine learning workflow. We considered reference samples derived by visual interpretation with spatial support of at least 30 m (Landsat and Sentinel), that could represent LULC classes for a point or region. For every dataset, we have an automated Python script to download the vector files and convert the original LULC classes into the following Global Pasture Watch Project classes: 0. Other land cover Natural and Semi-natural grassland Cultivated grassland Crops and other related agricultural practices We also empirically assigned a weight to each sample based on the original dataset's class descriptions, reflecting the level of mixture within the class. This weighting system ranged from 1 (Low) to 3 (High), with higher weights indicating a greater mixture of different landscapes within the same class. Samples with low mixture levels are more accurate and will be more effective for differentiating typologies and for validation purposes. The harmonized dataset includes these columns: Attribute Name Definition dataset_name Original dataset name reference_year Reference year of samples from the original dataset original_lulc_class LULC class from the original dataset gpw_lulc_class Global Pasture Watch LULC class sample_weight Sample's weight based on the level of mixture of landscapes compressed within the original LULC class Reclassification Process We developed a specifically tailored grassland ontology. This ontology serves as a reference system for classifying different grassland types, facilitating the harmonization of reference samples obtained from various sources. Our ontology likely focuses on differentiating between two primary grassland classes relevant to the project: cultivated grasslands (1) and natural or semi-natural grassland (2). Using the ontology as a guide, we reclassified samples from original datasets. Here's how we approached the different scenarios: Broader Definitions: For datasets with a single, broad classification level (e.g., grassland or herbaceous vegetation), we relied on the description provided within the dataset's metadata to determine the most appropriate mapping to our ontology's classes. In those cases, the level of mixture of different grassland typologies was higher, meaning the samples might not effectively differentiate between cultivated and natural grasslands. Multiple Classification Levels: Some datasets might have more detailed classifications. For example, a class named "Temporary grass crops" would likely be reclassified as "cultivated grassland (class 1)" based on the ontology's definition; while a class like "Savannah" would be reclassified as "natural grassland (class 2)". Level Class GPW class Description 5 GRASSLAND 1 or 2 Land (and the vegetation growing on it) devoted to the production of introduced or indigenous forage for harvest by grazing, cutting, or both. The vegetation of grassland in this context is broadly interpreted to include grasses, legumes and other forbs, and at times woody species may be present 5.1 Cultivated grassland 2 Forage is established with domesticated introduced or indigenous species that may receive periodic cultural treatment such as renovation, fertilization or weed control. 5.1.1 Annual grassland 2 Forage established annually, usually with annual plants, and generally involves soil disturbance, removal of existing vegetation, and other cultivation practices. 5.1.2 Temporary grassland 2 Land on which vegetation is composed of annual, biennial, or perennial forage species kept for a short period of time (usually only a few years). It can also be integrated in a crop rotation (ley). 5.2 Natural and Semi-Natural grassland 1 Areas dominated by herbaceous vegetation (grasses, sedges, rushes) with minimal woody plant cover, shaped by natural processes (fire, grazing by wild animals) or influenced by traditional human activities (grazing livestock, mowing) that maintain the natural character of the vegetation. 5.2.1 Campos 1 Grassland consisting mainly of grasses, along with herbs, small shrubs and occasional trees; on undulating and hilly landscape, with variable soil fertility. 5.2.2 LIanos 1 Extensive system of grasslands, seasonally flooded, with infertile and acidic soils. 5.2.3 Meadow 1 A natural or semi-natural grassland often associated with the conservation of hay or silage Full ontology here: https://docs.google.com/spreadsheets/d/1rRbM63flbizg6eP0PtogePM4K6ZnEDoFauCrV3TxZtA/edit?usp=sharing Reclassification Example MapBiomas Original WorldCereal Class Description (from Metadata) GPW Class Weight Justification Grassland Vegetation with a predominance of graminoid herbaceous stratum, including herbaceous and subshrub dicotyledons. The botanical composition is influenced by soil and topographic gradients as well as grazing management (livestock) 1 2 Definition emphasizes influence of grazing but doesn't specify planting. Potential for natural origin. Pasture Predominantly planted pasture areas, directly relatedto agricultural activity. Natural pasture areas, in turn, arepredominantly characterized as grasslands or wetland, and may or may not be subjected to grazing practices. 2 1 Highlights "planted" areas for grazing. Suggests active management practices. Look-up tables Datasets lookup tables https://docs.google.com/spreadsheets/d/1i9mQefIW_eOJNW29mf9EqB6BcZr1UAM0HbP9iCE1icI/edit?usp=sharing Quality and Assurance Analysis Our Quality Assurance (QA) analysis focused on data accuracy and consistency. We ensured the integrity of our dataset through the following approach: Duplicate Removal: We identified and excluded duplicate rows. This involved checking for rows with identical coordinates and corresponding years. These duplicates likely represent redundant entries within the original dataset or within different datasets. Outlier Detection: We screened for outliers by identifying data points with values exceeding the geographic boundaries of the original source's study area. Such values could indicate errors in data collection or processing within the original dataset. By eliminating duplicates and outliers, we ensure that the training and validation processes rely on a clean and accurate dataset. FILES Harmonizing the Datasets Add scripts Quality Assurance Analysis Add scrips RESULTS The final harmonized reference samples and documented processes are uploaded to Google Drive and GitHub for sharing and reproducibility. Link to the data on Google Drive Link to GitHub ACKNOWLEDGMENTS The development of this global inventory of reference samples and EO/gridded datasets relied on valuable contributions from various sources. We would like to express our sincere gratitude to the creators and maintainers of all datasets used in this project. LICENSE This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. REFERENCES Van Tricht, K. et al. Worldcereal: a dynamic open-source system for global-scale, seasonal, and reproducible crop and irrigation mapping. Earth Syst. Sci. Data 15, 5491–5515, 10.5194/essd-15-5491-2023 (2023) Schneider, M., Schelte, T., Schmitz, F. & Körner, M. Eurocrops: The largest harmonized open crop dataset across the european union. Sci. Data 10, 612, 10.1038/s41597-023-02517-0 (2023) Souza, C. M. et al. Reconstructing Three Decades of Land Use and Land Cover Changes in Brazilian Biomes with Landsat Archive and Earth Engine. Remote. Sens. 12, 2735, 10.3390/rs12172735 (2020) Stanimirova, R. et al. A global land cover training dataset from 1984 to 2020. Sci. Data 10, 879 (2023) d’Andrimont, R. et al. Harmonised lucas in-situ land cover and use database for field surveys from 2006 to 2018 in the european union. Sci. data 7, 352, 10.1038/s41597-019-0340-y (2020) Stehman, S. V., Pengra, B. W., Horton, J. A. & Wellington, D. F. Validation of the us geological survey’s land change monitoring, assessment and projection (lcmap) collection 1.0 annual land cover products 1985–2017. Remot Sensing environment 265, 112646, 10.1016/j.rse.2021.112646 (2021). Tsendbazar, N. et al. Product validation report (d12-pvr) v 1.1 (2021)