electricsheepafrica/africa-hdx-hapi-data-availability

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license: cc-by-4.0
multilinguality:
- monolingual
size_categories:
- 10K<n<100K
source_datasets:
- original
task_categories:
- other
task_ids: []
tags:
- africa
- humanitarian
- hdx
- electric-sheep-africa
- indicators
- afg
- alb
- dza
- asm
- and
pretty_name: "HDX HAPI - Metadata: Data Availability"
dataset_info:
splits:
- name: train
num_examples: 37420
- name: test
num_examples: 9355
---

# HDX HAPI - Metadata: Data Availability

**Publisher:** HDX Humanitarian API Data · **Source:** [HDX](https://data.humdata.org/dataset/hdx-hapi-data-availability) · **License:** `cc-by-igo` · **Updated:** 2026-02-18

---

## Abstract

This dataset contains data obtained from the
[HDX Humanitarian API](https://hapi.humdata.org/) (HDX HAPI),
which provides standardized humanitarian indicators designed
for seamless interoperability from multiple sources.
The data facilitates automated workflows and visualizations
to support humanitarian decision making.
For more information, please see the HDX HAPI
[landing page](https://data.humdata.org/hapi)
and
[documentation](https://hdx-hapi.readthedocs.io/en/latest/).

Each row in this dataset represents subnational administrative unit observations. Temporal coverage is indicated by the `hapi_updated_date` column(s). Geographic scope: **AFG, ALB, DZA, ASM, AND, AGO, AIA, ATA, and 237 others**.

*Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).*

---

## Dataset Characteristics

| | |
|---|---|
| **Domain** | Humanitarian and development data |
| **Unit of observation** | Subnational administrative unit observations |
| **Rows (total)** | 46,776 |
| **Columns** | 11 (1 numeric, 9 categorical, 1 datetime) |
| **Train split** | 37,420 rows |
| **Test split** | 9,355 rows |
| **Geographic scope** | AFG, ALB, DZA, ASM, AND, AGO, AIA, ATA, and 237 others |
| **Publisher** | HDX Humanitarian API Data |
| **HDX last updated** | 2026-02-18 |

---

## Variables

**Geographic** — `category` (geography-infrastructure, climate, coordination-context), `subcategory` (baseline-population, hazards-rainfall, conflict-events), `location_code` (MEX, COL, NGA), `admin1_code` (MX20, MX30, CO05), `admin1_name` (Oaxaca, Antioquia, Veracruz de Ignacio de la Llave) and 3 others.

**Temporal** — `hapi_updated_date`.

**Identifier / Metadata** — `esa_source` (HDX), `esa_processed` (2026-04-15).

---

## Quick Start

```python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-hdx-hapi-data-availability")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()
```

---

## Schema

| Column | Type | Null % | Range / Sample Values |
|---|---|---|---|
| `category` | object | 0.0% | geography-infrastructure, climate, coordination-context |
| `subcategory` | object | 0.0% | baseline-population, hazards-rainfall, conflict-events |
| `location_code` | object | 0.0% | MEX, COL, NGA |
| `admin1_code` | object | 3.3% | MX20, MX30, CO05 |
| `admin1_name` | object | 3.3% | Oaxaca, Antioquia, Veracruz de Ignacio de la Llave |
| `admin2_code` | object | 14.8% | CF422, MZ0906, ML0502 |
| `admin2_name` | object | 14.8% | Sucre, Bolívar, Libertador |
| `admin_level` | int64 | 0.0% | 0.0 – 2.0 (mean 1.8347) |
| `hapi_updated_date` | datetime64[ns] | 0.0% | |
| `esa_source` | object | 0.0% | HDX |
| `esa_processed` | object | 0.0% | 2026-04-15 |

---

## Numeric Summary

| Column | Min | Max | Mean | Median |
|---|---|---|---|---|
| `admin_level` | 0.0 | 2.0 | 1.8347 | 2.0 |

---

## Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 1 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

---

## Limitations

- Data originates from HDX Humanitarian API Data and has not been independently validated by ESA.
- Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
- This dataset spans 245 countries; geographic and methodological inconsistencies across national boundaries may affect cross-country comparability.
- Refer to the [original HDX dataset page](https://data.humdata.org/dataset/hdx-hapi-data-availability) for the publisher's own methodology notes and caveats.

---

## Citation

```bibtex
@dataset{hdx_africa_hdx_hapi_data_availability,
title = {HDX HAPI - Metadata: Data Availability},
author = {HDX Humanitarian API Data},
year = {2026},
url = {https://data.humdata.org/dataset/hdx-hapi-data-availability},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}
```

---

*[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*

annotations_creators: 无注释
language_creators: 现有公开资源采集
language: 英语
license: CC BY 4.0
multilinguality: 单语言
size_categories: 10000<n<100000
source_datasets: 原生数据集
task_categories: 其他
task_ids: 无
tags: 非洲、人道主义、HDX、Electric Sheep Africa、人道主义指标、AFG、ALB、DZA、ASM、AND
pretty_name: "HDX HAPI - 元数据：数据可用性"
dataset_info:
splits:
- name: train
num_examples: 37420
- name: test
num_examples: 9355

# HDX HAPI - 元数据：数据可用性

**发布方：** HDX人道主义API数据集 · **来源：** [HDX（Humanitarian Data Exchange，人道主义数据交换）](https://data.humdata.org/dataset/hdx-hapi-data-availability) · **许可证：** `cc-by-igo` · **最后更新：** 2026-02-18

---

## 摘要

本数据集包含从[HDX人道主义API（HDX HAPI）](https://hapi.humdata.org/)获取的数据，该接口提供来自多源的标准化人道主义指标，旨在实现跨源无缝互操作。数据集支持自动化工作流与可视化流程，助力人道主义决策工作。如需了解更多信息，请访问HDX HAPI的[首页](https://data.humdata.org/hapi)与[官方文档](https://hdx-hapi.readthedocs.io/en/latest/)。

本数据集的每一行均代表一个次国家级行政单元的观测数据，时间覆盖范围由`hapi_updated_date`列标注。地理覆盖范围：**AFG（阿富汗）、ALB（阿尔巴尼亚）、DZA（阿尔及利亚）、ASM（美属萨摩亚）、AND（安道尔）、AGO（安哥拉）、AIA（安圭拉）、ATA（南极洲）以及另外237个国家/地区**。

*本数据集已由[Electric Sheep Africa](https://huggingface.co/electricsheepafrica)整理为适用于机器学习的Parquet格式。*

---

## 数据集特征

| | |
|---|---|
| **领域** | 人道主义与发展数据 |
| **观测单元** | 次国家级行政单元 |
| **总行数** | 46776 |
| **列数** | 11列（1个数值型、9个分类型、1个日期时间型） |
| **训练集划分** | 37420行 |
| **测试集划分** | 9355行 |
| **地理覆盖范围** | AFG（阿富汗）、ALB（阿尔巴尼亚）、DZA（阿尔及利亚）、ASM（美属萨摩亚）、AND（安道尔）、AGO（安哥拉）、AIA（安圭拉）、ATA（南极洲）以及另外237个国家/地区 |
| **发布方** | HDX人道主义API数据集 |
| **HDX最后更新时间** | 2026-02-18 |

---

## 变量说明

**地理类变量**：包含`category`（类别：地理-基础设施、气候、协调-背景）、`subcategory`（子类别：基线人口、灾害-降雨、冲突事件）、`location_code`（位置代码：MEX（墨西哥）、COL（哥伦比亚）、NGA（尼日利亚））、`admin1_code`（一级行政代码：MX20、MX30、CO05）、`admin1_name`（一级行政区域名称：瓦哈卡州、安蒂奥基亚省、韦拉克鲁斯-伊格纳西奥-德拉拉韦州）以及另外3个变量。

**时间类变量**：`hapi_updated_date`。

**标识/元数据类变量**：`esa_source`（数据来源：HDX）、`esa_processed`（数据处理时间：2026-04-15）。

---

## 快速上手

python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-hdx-hapi-data-availability")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()

---

## 数据结构

| 列名 | 数据类型 | 空值占比 | 取值范围/示例值 |
|---|---|---|---|
| `category` | 字符串型 | 0.0% | 地理-基础设施、气候、协调-背景 |
| `subcategory` | 字符串型 | 0.0% | 基线人口、灾害-降雨、冲突事件 |
| `location_code` | 字符串型 | 0.0% | MEX（墨西哥）、COL（哥伦比亚）、NGA（尼日利亚） |
| `admin1_code` | 字符串型 | 3.3% | MX20、MX30、CO05 |
| `admin1_name` | 字符串型 | 3.3% | 瓦哈卡州、安蒂奥基亚省、韦拉克鲁斯-伊格纳西奥-德拉拉韦州 |
| `admin2_code` | 字符串型 | 14.8% | CF422、MZ0906、ML0502 |
| `admin2_name` | 字符串型 | 14.8% | 苏克雷市、玻利瓦尔市、解放者市 |
| `admin_level` | 64位整型 | 0.0% | 0.0~2.0（均值为1.8347） |
| `hapi_updated_date` | 日期时间型 | 0.0% | 无 |
| `esa_source` | 字符串型 | 0.0% | HDX |
| `esa_processed` | 字符串型 | 0.0% | 2026-04-15 |

---

## 数值型变量统计

| 列名 | 最小值 | 最大值 | 均值 | 中位数 |
|---|---|---|---|---|
| `admin_level` | 0.0 | 2.0 | 1.8347 | 2.0 |

---

## 数据整理流程

原始数据通过CKAN（Comprehensive Knowledge Archive Network，综合知识存档网络）API从HDX下载，并转换为Parquet格式。列名统一转换为小写，并规范化为蛇形命名法（snake_case）。常见的缺失值标记（`N/A`、`null`、`none`、`-`、`unknown`、`no data`、`#N/A`）均统一替换为`NaN`。根据解析成功率（阈值为85%以上），将1列从字符串类型转换为数值型或日期时间型。本数据集使用固定随机种子（42）按照80/20的比例划分为训练集与测试集，并保存为Snappy压缩的Parquet格式文件。

---

## 局限性说明

- 本数据集的数据来源于HDX人道主义API数据集，未经过Electric Sheep Africa（ESA）的独立验证。
- 自动化清洗流程无法修正原始数据集中的错报值、定义不一致问题或采样偏差。
- 本数据集覆盖245个国家/地区；各国间的地理与方法学差异可能会影响跨国数据的可比性。
- 如需了解发布方的方法学说明与注意事项，请参阅[HDX原始数据集页面](https://data.humdata.org/dataset/hdx-hapi-data-availability)。

---

## 引用格式

bibtex
@dataset{hdx_africa_hdx_hapi_data_availability,
title = {HDX HAPI - Metadata: Data Availability},
author = {HDX Humanitarian API Data},
year = {2026},
url = {https://data.humdata.org/dataset/hdx-hapi-data-availability},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}

---

*[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — 非洲机器学习数据集基础设施提供商，尼日利亚拉各斯。*