electricsheepafrica/africa-hdx-hapi-esh

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license: cc-by-4.0
multilinguality:
- monolingual
size_categories:
- 1K<n<10K
source_datasets:
- original
task_categories:
- other
task_ids: []
tags:
- africa
- humanitarian
- hdx
- electric-sheep-africa
- indicators
- refugees
- returnees
- esh
pretty_name: "HDX HAPI Data for Western Sahara"
dataset_info:
splits:
- name: train
num_examples: 3016
- name: test
num_examples: 754
---

# HDX HAPI Data for Western Sahara

**Publisher:** HDX Humanitarian API Data · **Source:** [HDX](https://data.humdata.org/dataset/hdx-hapi-esh) · **License:** `cc-by-igo` · **Updated:** 2026-02-18

---

## Abstract

This dataset contains data obtained from the
[HDX Humanitarian API](https://hapi.humdata.org/) (HDX HAPI),
which provides standardized humanitarian indicators designed
for seamless interoperability from multiple sources.
The data facilitates automated workflows and visualizations
to support humanitarian decision making.
For more information, please see the HDX HAPI
[landing page](https://data.humdata.org/hapi)
and
[documentation](https://hdx-hapi.readthedocs.io/en/latest/).

Each row in this dataset represents geolocated point observations. Temporal coverage is indicated by the `reference_period_start`, `reference_period_end` column(s). Geographic scope: **ESH**.

*Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).*

---

## Dataset Characteristics

| | |
|---|---|
| **Domain** | Demographics and population |
| **Unit of observation** | Geolocated point observations |
| **Rows (total)** | 3,770 |
| **Columns** | 16 (3 numeric, 7 categorical, 2 datetime) |
| **Train split** | 3,016 rows |
| **Test split** | 754 rows |
| **Geographic scope** | ESH |
| **Publisher** | HDX Humanitarian API Data |
| **HDX last updated** | 2026-02-18 |

---

## Variables

**Geographic** — `origin_location_code` (ESH), `asylum_location_code` (CHE, FRA, AGO), `asylum_has_hrp`, `asylum_in_gho`, `population_group` (REF, ASY, OOC) and 2 others.

**Temporal** — `reference_period_start`, `reference_period_end`.

**Demographic** — `gender` (f, m, all), `age_range` (all, 0-4, 5-11), `min_age` (range 0.0–60.0).

**Identifier / Metadata** — `esa_source` (HDX), `esa_processed` (2026-04-06).

**Other** — `origin_has_hrp`, `origin_in_gho`.

---

## Quick Start

```python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-hdx-hapi-esh")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()
```

---

## Schema

| Column | Type | Null % | Range / Sample Values |
|---|---|---|---|
| `origin_location_code` | object | 0.0% | ESH |
| `origin_has_hrp` | bool | 0.0% | |
| `origin_in_gho` | bool | 0.0% | |
| `asylum_location_code` | object | 0.0% | CHE, FRA, AGO |
| `asylum_has_hrp` | bool | 0.0% | |
| `asylum_in_gho` | bool | 0.0% | |
| `population_group` | object | 0.0% | REF, ASY, OOC |
| `gender` | object | 0.0% | f, m, all |
| `age_range` | object | 0.0% | all, 0-4, 5-11 |
| `min_age` | float64 | 23.1% | 0.0 – 60.0 (mean 19.0) |
| `max_age` | float64 | 38.5% | 4.0 – 59.0 (mean 22.75) |
| `population` | int64 | 0.0% | 0.0 – 173600.0 (mean 1281.352) |
| `reference_period_start` | datetime64[ns] | 0.0% | |
| `reference_period_end` | datetime64[ns] | 0.0% | |
| `esa_source` | object | 0.0% | HDX |
| `esa_processed` | object | 0.0% | 2026-04-06 |

---

## Numeric Summary

| Column | Min | Max | Mean | Median |
|---|---|---|---|---|
| `min_age` | 0.0 | 60.0 | 19.0 | 12.0 |
| `max_age` | 4.0 | 59.0 | 22.75 | 14.0 |
| `population` | 0.0 | 173600.0 | 1281.352 | 0.0 |

---

## Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 2 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

---

## Limitations

- Data originates from HDX Humanitarian API Data and has not been independently validated by ESA.
- Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
- The following columns have >20% missing values and should be treated with caution in modelling: `min_age`, `max_age`.
- Refer to the [original HDX dataset page](https://data.humdata.org/dataset/hdx-hapi-esh) for the publisher's own methodology notes and caveats.

---

## Citation

```bibtex
@dataset{hdx_africa_hdx_hapi_esh,
title = {HDX HAPI Data for Western Sahara},
author = {HDX Humanitarian API Data},
year = {2026},
url = {https://data.humdata.org/dataset/hdx-hapi-esh},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}
```

---

*[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*

annotations_creators:
- 无标注
language_creators:
- 现有资源获取
language:
- 英语
license:
- CC BY 4.0
multilinguality:
- 单语言
size_categories:
- 1000 < n < 10000
source_datasets:
- 原生数据集
task_categories:
- 其他
task_ids: []
tags:
- 非洲
- 人道主义
- HDX（Humanitarian Data Exchange）
- Electric Sheep Africa（电羊非洲）
- 指标
- 难民
- 返回人员
- ESH
pretty_name: "撒哈拉西南地区HDX HAPI数据集"
dataset_info:
splits:
- name: train
num_examples: 3016
- name: test
num_examples: 754

# 撒哈拉西南地区HDX HAPI数据集

**发布方:** HDX人道主义API数据 · **来源:** [HDX（Humanitarian Data Exchange）](https://data.humdata.org/dataset/hdx-hapi-esh) · **许可证:** `cc-by-igo` · **更新:** 2026-02-18

---

## 摘要

本数据集包含从[HDX人道主义API（HDX HAPI）](https://hapi.humdata.org/)获取的数据，该接口提供标准化的人道主义指标，旨在实现多源数据的无缝互操作。本数据集支持自动化工作流与可视化，以辅助人道主义决策制定。如需更多信息，请参阅HDX HAPI的[着陆页](https://data.humdata.org/hapi)与[官方文档](https://hdx-hapi.readthedocs.io/en/latest/)。

每一行代表一个地理定位的点位观测值。时间覆盖范围由`reference_period_start`（参考周期开始时间）与`reference_period_end`（参考周期结束时间）列标注。地理范围：**ESH（撒哈拉西南地区）**。

*本数据集由[Electric Sheep Africa（电羊非洲）](https://huggingface.co/electricsheepafrica)整理为机器学习可用的Parquet列存格式（Parquet）。*

---

## 数据集特征

| | |
|---|---|
| **领域** | 人口与人口统计学 |
| **观测单元** | 地理定位点位观测值 |
| **总样本行数** | 3,770 |
| **列数** | 16列（其中3列为数值型、7列为分类型、2列为日期时间型） |
| **训练集划分** | 3,016行 |
| **测试集划分** | 754行 |
| **地理范围** | ESH |
| **发布方** | HDX人道主义API数据 |
| **HDX更新时间** | 2026-02-18 |

---

## 变量说明

**地理类变量** — `origin_location_code`（来源地位置代码，取值为ESH）、`asylum_location_code`（庇护地位置代码，取值包括CHE、FRA、AGO）、`asylum_has_hrp`、`asylum_in_gho`、`population_group`（人口群体，取值包括REF、ASY、OOC）及另外2个变量。

**时间类变量** — `reference_period_start`（参考周期开始时间）、`reference_period_end`（参考周期结束时间）。

**人口统计类变量** — `gender`（性别，取值包括f、m、all）、`age_range`（年龄区间，取值包括all、0-4、5-11）、`min_age`（最小年龄，取值范围0.0–60.0）。

**标识符/元数据类变量** — `esa_source`（数据来源，取值为HDX）、`esa_processed`（数据处理时间，取值为2026-04-06）。

**其他变量** — `origin_has_hrp`、`origin_in_gho`。

---

## 快速上手

python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-hdx-hapi-esh")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()

---

## 数据Schema

| 列名 | 数据类型 | 缺失率 | 取值范围/示例值 |
|---|---|---|---|
| `origin_location_code` | 对象型 | 0.0% | ESH |
| `origin_has_hrp` | 布尔型 | 0.0% | 无 |
| `origin_in_gho` | 布尔型 | 0.0% | 无 |
| `asylum_location_code` | 对象型 | 0.0% | CHE、FRA、AGO |
| `asylum_has_hrp` | 布尔型 | 0.0% | 无 |
| `asylum_in_gho` | 布尔型 | 0.0% | 无 |
| `population_group` | 对象型 | 0.0% | REF、ASY、OOC |
| `gender` | 对象型 | 0.0% | f、m、all |
| `age_range` | 对象型 | 0.0% | all、0-4、5-11 |
| `min_age` | 双精度浮点型 | 23.1% | 0.0 – 60.0（均值为19.0） |
| `max_age` | 双精度浮点型 | 38.5% | 4.0 – 59.0（均值为22.75） |
| `population` | 64位整型 | 0.0% | 0.0 – 173600.0（均值为1281.352） |
| `reference_period_start` | 纳秒级日期时间型 | 0.0% | 无 |
| `reference_period_end` | 纳秒级日期时间型 | 0.0% | 无 |
| `esa_source` | 对象型 | 0.0% | HDX |
| `esa_processed` | 对象型 | 0.0% | 2026-04-06 |

---

## 数值型变量统计摘要

| 列名 | 最小值 | 最大值 | 均值 | 中位数 |
|---|---|---|---|---|
| `min_age` | 0.0 | 60.0 | 19.0 | 12.0 |
| `max_age` | 4.0 | 59.0 | 22.75 | 14.0 |
| `population` | 0.0 | 173600.0 | 1281.352 | 0.0 |

---

## 数据整理流程

原始数据通过CKAN API从HDX下载，并转换为Parquet列存格式。列名统一转换为小写并标准化为蛇形命名法（snake_case）。通用缺失值标记（`N/A`、`null`、`none`、`-`、`unknown`、`no data`、`#N/A`）被统一替换为`NaN`。基于解析成功率（阈值>85%），将2列从字符串类型转换为数值型或日期时间型。本数据集以固定随机种子（42）按80/20比例划分为训练集与测试集，并保存为Snappy压缩（Snappy）的Parquet格式。

---

## 局限性

- 数据源自HDX人道主义API数据集，未经过Electric Sheep Africa（电羊非洲）的独立验证。
- 自动化清洗无法修正原始数据收集中的错报值、定义不一致或抽样偏差问题。
- 以下列的缺失率超过20%，在建模时需谨慎使用：`min_age`、`max_age`。
- 请参阅[原始HDX数据集页面](https://data.humdata.org/dataset/hdx-hapi-esh)获取发布方的方法论说明与注意事项。

---

## 引用格式

bibtex
@dataset{hdx_africa_hdx_hapi_esh,
title = {HDX HAPI Data for Western Sahara},
author = {HDX Humanitarian API Data},
year = {2026},
url = {https://data.humdata.org/dataset/hdx-hapi-esh},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}

---

*[Electric Sheep Africa（电羊非洲）](https://huggingface.co/electricsheepafrica) — 非洲的机器学习数据集基础设施。尼日利亚拉各斯。*