electricsheepafrica/africa-climada-crop-production-dataset
收藏资源简介：
--- annotations_creators: - no-annotation language_creators: - found language: - en license: cc-by-4.0 multilinguality: - monolingual size_categories: - 1K<n<10K source_datasets: - original task_categories: - other task_ids: [] tags: - africa - humanitarian - hdx - electric-sheep-africa - geodata - hazards-and-risk - humanitarian-response-plan-hrp - hxl - afg - bfa - bdi - cmr - caf pretty_name: "Crop production: Humanitarian Response Plan (HRP) Countries Exposure Data for Disaster Risk Assessment" dataset_info: splits: - name: train num_examples: 2650 - name: test num_examples: 662 --- # Crop production: Humanitarian Response Plan (HRP) Countries Exposure Data for Disaster Risk Assessment **Publisher:** ETH Zürich - Weather and Climate Risks · **Source:** [HDX](https://data.humdata.org/dataset/climada-crop-production-dataset) · **License:** `cc-by` · **Updated:** 2025-09-03 --- ## Abstract Historical and twenty-first century crop production in tons. Global gridded (4km resolution) crop yield simulations for maize, rice, soybean, and wheat, encompassing an ensemble of transient yield simulation output from eight global gridded crop models driven by bias-corrected output from five global climate models, as facilitated by the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP, isimip.org) Each row in this dataset represents first-level administrative unit observations. Data was last updated on HDX on 2025-09-03. Geographic scope: **AFG, BFA, BDI, CMR, CAF, TCD, COL, COD, and 15 others**. *Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).* --- ## Dataset Characteristics | | | |---|---| | **Domain** | Climate and environment | | **Unit of observation** | First-level administrative unit observations | | **Rows (total)** | 3,313 | | **Columns** | 9 (3 numeric, 6 categorical, 0 datetime) | | **Train split** | 2,650 rows | | **Test split** | 662 rows | | **Geographic scope** | AFG, BFA, BDI, CMR, CAF, TCD, COL, COD, and 15 others | | **Publisher** | ETH Zürich - Weather and Climate Risks | | **HDX last updated** | 2025-09-03 | --- ## Variables **Geographic** — `country_name` (Nigeria, Afghanistan, Colombia), `admin1_name` (Centre, Sucre, North), `latitude` (range -25.96–51.4062), `longitude` (range -81.55–98.7083). **Outcome / Measurement** — `value` (range 0.0–1521161858.0). **Identifier / Metadata** — `esa_source` (HDX), `esa_processed` (2026-04-04). **Other** — `aggregation` (sum), `indicator` (crop-production.mai.noirr.USD, crop-production.mai.firr.USD, crop-production.whe.noirr.USD). --- ## Quick Start ```python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-climada-crop-production-dataset") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() ``` --- ## Schema | Column | Type | Null % | Range / Sample Values | |---|---|---|---| | `country_name` | object | 0.0% | Nigeria, Afghanistan, Colombia | | `admin1_name` | object | 0.0% | Centre, Sucre, North | | `latitude` | float64 | 0.0% | -25.96 – 51.4062 (mean 13.6832) | | `longitude` | float64 | 0.0% | -81.55 – 98.7083 (mean 14.9992) | | `aggregation` | object | 0.0% | sum | | `indicator` | object | 0.0% | crop-production.mai.noirr.USD, crop-production.mai.firr.USD, crop-production.whe.noirr.USD | | `value` | float64 | 0.0% | 0.0 – 1521161858.0 (mean 7654114.5305) | | `esa_source` | object | 0.0% | HDX | | `esa_processed` | object | 0.0% | 2026-04-04 | --- ## Numeric Summary | Column | Min | Max | Mean | Median | |---|---|---|---|---| | `latitude` | -25.96 | 51.4062 | 13.6832 | 10.5663 | | `longitude` | -81.55 | 98.7083 | 14.9992 | 24.9758 | | `value` | 0.0 | 1521161858.0 | 7654114.5305 | 0.0 | --- ## Curation Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 3 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet. --- ## Limitations - Data originates from ETH Zürich - Weather and Climate Risks and has not been independently validated by ESA. - Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection. - This dataset spans 23 countries; geographic and methodological inconsistencies across national boundaries may affect cross-country comparability. - Refer to the [original HDX dataset page](https://data.humdata.org/dataset/climada-crop-production-dataset) for the publisher's own methodology notes and caveats. --- ## Citation ```bibtex @dataset{hdx_africa_climada_crop_production_dataset, title = {Crop production: Humanitarian Response Plan (HRP) Countries Exposure Data for Disaster Risk Assessment}, author = {ETH Zürich - Weather and Climate Risks}, year = {2025}, url = {https://data.humdata.org/dataset/climada-crop-production-dataset}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } ``` --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*
annotations_creators: - 无标注 language_creators: - 公开数据源采集 language: - en license: cc-by-4.0 multilinguality: - 单语言 size_categories: - 1000<n<10000 source_datasets: - 原创数据集 task_categories: - 其他 task_ids: [] tags: - 非洲 - 人道主义 - HDX - electric-sheep-africa - 地理数据 - 灾害与风险 - 人道主义响应计划(HRP) - hxl - afg - bfa - bdi - cmr - caf pretty_name: "作物生产：用于灾害风险评估的人道主义响应计划(HRP)国家暴露数据" dataset_info: splits: - name: train num_examples: 2650 - name: test num_examples: 662 --- # 作物生产：用于灾害风险评估的人道主义响应计划(HRP)国家暴露数据 **发布方**：苏黎世联邦理工学院-天气与气候风险研究团队 · **数据源**：[HDX](https://data.humdata.org/dataset/climada-crop-production-dataset) · **授权协议**：`cc-by` · **更新时间**：2025-09-03 --- ## 摘要 本数据集包含历史及21世纪作物产量数据（单位：吨）。涵盖玉米、水稻、大豆与小麦的全球网格化（分辨率4km）作物产量模拟结果，包含由5个全球气候模型的偏差校正输出驱动的8个全球网格化作物模型的瞬态产量模拟集合结果，相关工作由跨部门影响模型比对项目(Inter-Sectoral Impact Model Intercomparison Project, ISIMIP, isimip.org)支持。 本数据集的每一行代表一级行政单元的观测数据。数据最近一次在HDX平台的更新时间为2025-09-03。地理覆盖范围：**AFG、BFA、BDI、CMR、CAF、TCD、COL、COD及另外15个国家**。 *本数据集已由[Electric Sheep Africa](https://huggingface.co/electricsheepafrica)整理为适用于机器学习的Parquet格式。* --- ## 数据集特征 | | | |---|---| | **领域** | 气候与环境 | | **观测单元** | 一级行政单元观测数据 | | **总数据行数** | 3,313 | | **列数** | 9（3个数值型、6个分类型、0个日期时间型） | | **训练集划分** | 2,650条 | | **测试集划分** | 662条 | | **地理覆盖范围** | AFG, BFA, BDI, CMR, CAF, TCD, COL, COD及另外15个国家 | | **发布方** | 苏黎世联邦理工学院-天气与气候风险研究团队 | | **HDX平台最后更新时间** | 2025-09-03 | --- ## 变量 **地理类变量**：`country_name`（国家名称，例如尼日利亚、阿富汗、哥伦比亚）、`admin1_name`（一级行政区域名称，例如中央省、苏克雷省、北部省）、`latitude`（纬度范围：-25.96~51.4062）、`longitude`（经度范围：-81.55~98.7083）。 **结果/测量类变量**：`value`（数值范围：0.0~1521161858.0）。 **标识符/元数据类变量**：`esa_source`（数据来源，例如HDX）、`esa_processed`（数据处理时间，例如2026-04-04）。 **其他变量**：`aggregation`（聚合方式，例如求和）、`indicator`（指标名称，例如crop-production.mai.noirr.USD、crop-production.mai.firr.USD、crop-production.whe.noirr.USD）。 --- ## 快速上手 python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-climada-crop-production-dataset") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() --- ## 数据结构 | 列名 | 数据类型 | 缺失率 | 范围/示例值 | |---|---|---|---| | `country_name` | 字符串型(object) | 0.0% | 尼日利亚、阿富汗、哥伦比亚 | | `admin1_name` | 字符串型 | 0.0% | 中央省、苏克雷省、北部省 | | `latitude` | 浮点型(float64) | 0.0% | -25.96 ~ 51.4062（均值13.6832） | | `longitude` | 浮点型 | 0.0% | -81.55 ~ 98.7083（均值14.9992） | | `aggregation` | 字符串型 | 0.0% | sum（求和） | | `indicator` | 字符串型 | 0.0% | crop-production.mai.noirr.USD、crop-production.mai.firr.USD、crop-production.whe.noirr.USD | | `value` | 浮点型 | 0.0% | 0.0 ~ 1521161858.0（均值7654114.5305） | | `esa_source` | 字符串型 | 0.0% | HDX | | `esa_processed` | 字符串型 | 0.0% | 2026-04-04 | --- ## 数值型变量统计摘要 | 列名 | 最小值 | 最大值 | 均值 | 中位数 | |---|---|---|---|---| | `latitude` | -25.96 | 51.4062 | 13.6832 | 10.5663 | | `longitude` | -81.55 | 98.7083 | 14.9992 | 24.9758 | | `value` | 0.0 | 1521161858.0 | 7654114.5305 | 0.0 | --- ## 数据整理流程 原始数据通过CKAN API从HDX平台下载，并转换为Parquet格式。列名统一转换为小写并标准化为蛇形命名法。常见缺失值标记（`N/A`、`null`、`none`、`-`、`unknown`、`no data`、`#N/A`）被统一替换为`NaN`。根据解析成功率（阈值>85%），将3列从字符串类型转换为数值型或日期时间型。本数据集以固定随机种子(42)按80/20比例划分为训练集与测试集，并保存为Snappy压缩的Parquet格式文件。 --- ## 数据集局限性 - 数据源自苏黎世联邦理工学院-天气与气候风险研究团队，未经过Electric Sheep Africa的独立验证。 - 自动化数据清洗无法修正原始数据收集中的错报值、定义不一致或抽样偏差问题。 - 本数据集覆盖23个国家，各国间的地理与方法学差异可能影响跨国数据的可比性。 - 请参阅[原始HDX数据集页面](https://data.humdata.org/dataset/climada-crop-production-dataset)获取发布方提供的方法学说明与免责声明。 --- ## 引用格式 bibtex @dataset{hdx_africa_climada_crop_production_dataset, title = {Crop production: Humanitarian Response Plan (HRP) Countries Exposure Data for Disaster Risk Assessment}, author = {ETH Zürich - Weather and Climate Risks}, year = {2025}, url = {https://data.humdata.org/dataset/climada-crop-production-dataset}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — 非洲机器学习数据集基础设施。尼日利亚拉各斯。*
