MCCN Case Study 2 - Spatial projection via modelled data

The MCCN project is to deliver tools to assist the agricultural sector to understand crop-environment relationships, specifically by facilitating generation of data cubes for spatiotemporal data. This repository contains Jupyter notebooks to demonstrate the functionality of the MCCN data cube components.The dataset contains input files for the case study (source_data), RO-Crate metadata (ro-crate-metadata.json), results from the case study (results), and Jupyter Notebook (MCCN-CASE 2.ipynb)Research Activity Identifier (RAiD)RAiD: https://doi.org/10.26292/8679d473Case StudiesThis repository contains code and sample data for the following case studies. Note that the analyses here are to demonstrate the software and result should not be considered scientifically or statistically meaningful. No effort has been made to address bias in samples, and sample data may not be available at sufficient density to warrant analysis. All case studies end with generation of an RO-Crate data package including the source data, the notebook and generated outputs, including netcdf exports of the datacubes themselves.Case Study 2 - Spatial projection via modelled dataDescriptionEstimate soil pH and electrical conductivity at 45 cm depth across a farm based on values collected from soil samples. This study demonstrates: 1) Description of spatial assets using STAC, 2) Loading heterogeneous data sources into a cube, 3) Spatial projection in xarray using different algorithms offered by the pykrige and rioxarray packages.Data sourcesBradGinns_SOIL2004_SoilData.csv - Soil measurements from the University of Sydney Llara Campey farm site from 2004, corresponding to sites L1, L3 and L4 describing mid-depth, soil apparent electrical conductivity (ECa), GammaK, Clay, Silt, Sand, pH and soil electrical conductivity (EC)Llara_Campey_field_boundaries_poly.shp - Field boundary shapes for the University of Sydney Llara Campey farm siteDependenciesThis notebook requires Python 3.10 or higherInstall relevant Python libraries with: pip install mccn-engine rocrate rioxarray pykrigeInstalling mccn-engine will install other dependenciesOverviewSelect soil sample measurements for pH or EC at 45 cm depthSplit sample measurements into 80% subset to model interpolated layers and 20% to test interpolated layersGenerate STAC metadata for layersLoad data cubeInterpolate pH and EC across site using the 80% subset and three different 2D interpolation methods from rioxarray (nearest, linear and cubic) and one from pykrige (linear)Calculate the error between each layer of interpolated values and measured values for the 20% setaside for testingCompare the mean and standard deviation of the errors for each interpolation methodClean up and package results as RO-CrateNotesThe granularity of variability in soil data significantly compromises all methodsDepending on the 80/20 split, different methods may appear more reliable, but the pykrige linear method is most often bestMCCN项目旨在研发工具，以辅助农业领域解析作物与环境间的相互关系，具体而言，可助力生成时空数据（spatiotemporal data）的数据立方体（data cube）。本仓库包含Jupyter笔记本（Jupyter Notebook），用于演示MCCN数据立方体组件的功能。 本数据集包含案例研究的输入文件（source_data）、RO-Crate元数据（ro-crate-metadata.json）、案例研究的结果（results）以及Jupyter笔记本（MCCN-CASE 2.ipynb）。研究活动标识符（Research Activity Identifier，RAiD）：https://doi.org/10.26292/8679d473 ## 案例研究 本仓库包含以下案例研究的代码与示例数据。请注意，此处的分析仅用于演示软件功能，所得结果不具备科学或统计学意义。本次研究未对样本偏差进行处理，且示例数据的密度可能不足以支撑有效的分析。所有案例研究均以生成RO-Crate数据包收尾，该数据包包含源数据、笔记本代码及生成的输出结果，其中还包括数据立方体的NetCDF格式导出文件。 ### 案例研究2——基于建模数据的空间投影 #### 说明 本案例基于土壤样本采集值，估算某农场45厘米深度处的土壤pH值与电导率（electrical conductivity）。本案例演示了以下内容：1）利用STAC（SpatioTemporal Asset Catalog）描述空间资产；2）将异构数据源加载至数据立方体中；3）借助pykrige与rioxarray库提供的多种算法，在xarray中实现空间投影。 #### 数据源 BradGinns_SOIL2004_SoilData.csv——2004年悉尼大学Llara Campey农场的土壤测量数据，对应L1、L3、L4三个采样点，包含中层土壤表观电导率（soil apparent electrical conductivity，ECa）、伽马钾（GammaK）、黏粒含量（Clay）、粉粒含量（Silt）、砂粒含量（Sand）、pH值与土壤电导率（EC）。 Llara_Campey_field_boundaries_poly.shp——悉尼大学Llara Campey农场的田界矢量形状文件。 #### 依赖项 本笔记本需使用Python 3.10及以上版本。可通过以下命令安装所需Python库：pip install mccn-engine rocrate rioxarray pykrige。安装mccn-engine时将自动安装其余依赖项。 #### 流程概述 1. 选取45厘米深度处的土壤pH值或电导率样本测量值。 2. 将样本测量值划分为80%的建模子集（用于生成插值图层）与20%的测试子集（用于验证插值图层）。 3. 为各图层生成STAC元数据。 4. 加载数据立方体。 5. 利用80%的建模子集，分别采用rioxarray提供的3种二维插值方法（最邻近、线性、三次插值）与pykrige提供的1种线性插值方法，对农场范围内的pH值与电导率进行空间插值。 6. 针对预留的20%测试子集，计算各插值图层的插值结果与实测值之间的误差。 7. 对比各插值方法的误差均值与标准差。 8. 整理结果并将其打包为RO-Crate数据包。 #### 注意事项 1. 土壤数据的变异性粒度会显著影响所有插值方法的效果。 2. 根据80/20的划分比例不同，不同方法的表现可靠性可能存在差异，但pykrige线性插值方法通常表现最优。