electricsheepafrica/africa-darfur-damaged-and-destroyed-villages
收藏资源简介：
--- annotations_creators: - no-annotation language_creators: - found language: - en license: other multilinguality: - monolingual size_categories: - 1K<n<10K source_datasets: - original task_categories: - tabular-classification - other task_ids: [] tags: - africa - humanitarian - hdx - electric-sheep-africa - conflict-violence - eastern-africa - geodata - sdn pretty_name: "Darfur Damaged and Destroyed Villages" dataset_info: splits: - name: train num_examples: 7346 - name: test num_examples: 1836 --- # Darfur Damaged and Destroyed Villages **Publisher:** U.S. Department of State - Humanitarian Information Unit (inactive) · **Source:** [HDX](https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages) · **License:** `other-pd-nr` · **Updated:** 2025-05-05 --- ## Abstract The Darfur Damaged and Destroyed Villages dataset describes the condition of villages in the Darfur region of Sudan that the U.S. Government has confirmed as either damaged or destroyed between the time period February 2003 to December 2010. Additionally, villages that are confirmed to have No Damage are also reported. Each row in this dataset represents geolocated point observations. Data was last updated on HDX on 2025-05-05. Geographic scope: **SDN**. *Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).* --- ## Dataset Characteristics | | | |---|---| | **Domain** | Conflict and security | | **Unit of observation** | Geolocated point observations | | **Rows (total)** | 9,183 | | **Columns** | 14 (9 numeric, 5 categorical, 0 datetime) | | **Train split** | 7,346 rows | | **Test split** | 1,836 rows | | **Geographic scope** | SDN | | **Publisher** | U.S. Department of State - Humanitarian Information Unit (inactive) | | **HDX last updated** | 2025-05-05 | --- ## Variables **Geographic** — `latitude` (range 91624.0–14243299.0), `lat_dd` (range 9.2733–15.5889), `longitude` (range 215006.0–25534651.0), `long_dd` (range 21.835–27.39), `yr_confirm` (range 2003.0–2010.0) and 4 others. **Identifier / Metadata** — `name` (HASHABA, KUMA, HILLET IBRAHIM), `esa_source` (HDX), `esa_processed` (2026-04-05). **Other** — `status` (NO DAMAGE, DESTROYED, DAMAGED), `structures` ( , ~25 of ~25, ~50 of ~50). --- ## Quick Start ```python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-darfur-damaged-and-destroyed-villages") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() ``` --- ## Schema | Column | Type | Null % | Range / Sample Values | |---|---|---|---| | `name` | object | 68.8% | HASHABA, KUMA, HILLET IBRAHIM | | `latitude` | float64 | 1.5% | 91624.0 – 14243299.0 (mean 150722.5973) | | `lat_dd` | float64 | 0.0% | 9.2733 – 15.5889 (mean 12.8066) | | `longitude` | float64 | 1.5% | 215006.0 – 25534651.0 (mean 289943.1203) | | `long_dd` | float64 | 0.0% | 21.835 – 27.39 (mean 24.4526) | | `status` | object | 0.0% | NO DAMAGE, DESTROYED, DAMAGED | | `structures` | object | 58.7% | , ~25 of ~25, ~50 of ~50 | | `yr_confirm` | float64 | 71.8% | 2003.0 – 2010.0 (mean 2004.7153) | | `aprox_str1` | float64 | 78.1% | 5.0 – 1500.0 (mean 105.8607) | | `aprox_str2` | float64 | 70.5% | 10.0 – 4000.0 (mean 117.8581) | | `yr_range1` | float64 | 66.3% | 2003.0 – 2010.0 (mean 2004.6276) | | `yr_range2` | float64 | 66.3% | 2003.0 – 2010.0 (mean 2004.7904) | | `esa_source` | object | 0.0% | HDX | | `esa_processed` | object | 0.0% | 2026-04-05 | --- ## Numeric Summary | Column | Min | Max | Mean | Median | |---|---|---|---|---| | `latitude` | 91624.0 | 14243299.0 | 150722.5973 | 125656.0 | | `lat_dd` | 9.2733 | 15.5889 | 12.8066 | 12.9725 | | `longitude` | 215006.0 | 25534651.0 | 289943.1203 | 243237.0 | | `long_dd` | 21.835 | 27.39 | 24.4526 | 24.5558 | | `yr_confirm` | 2003.0 | 2010.0 | 2004.7153 | 2004.0 | | `aprox_str1` | 5.0 | 1500.0 | 105.8607 | 50.0 | | `aprox_str2` | 10.0 | 4000.0 | 117.8581 | 75.0 | | `yr_range1` | 2003.0 | 2010.0 | 2004.6276 | 2004.0 | | `yr_range2` | 2003.0 | 2010.0 | 2004.7904 | 2004.0 | --- ## Curation Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 1 column(s) with >80% missing values were removed: `sec_town`. 3 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet. --- ## Limitations - Data originates from U.S. Department of State - Humanitarian Information Unit (inactive) and has not been independently validated by ESA. - Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection. - The following columns have >20% missing values and should be treated with caution in modelling: `name`, `structures`, `yr_confirm`, `aprox_str1`, `aprox_str2`, `yr_range1`, `yr_range2`. - Refer to the [original HDX dataset page](https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages) for the publisher's own methodology notes and caveats. --- ## Citation ```bibtex @dataset{hdx_africa_darfur_damaged_and_destroyed_villages, title = {Darfur Damaged and Destroyed Villages}, author = {U.S. Department of State - Humanitarian Information Unit (inactive)}, year = {2025}, url = {https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } ``` --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*
annotations_creators: - 无注释 language_creators: - 公开采集 language: - 英语 license: - 其他 multilinguality: - 单语种 size_categories: - 1000 < 样本数 < 10000 source_datasets: - 原创数据集 task_categories: - 表格分类 - 其他 task_ids: - 无 tags: - 非洲 - 人道主义 - HDX - electric-sheep-africa - 冲突与暴力 - 东非 - 地理数据 - SDN pretty_name: "达尔富尔受损与损毁村庄数据集" dataset_info: splits: - name: 训练集 num_examples: 7346 - name: 测试集 num_examples: 1836 --- # 达尔富尔受损与损毁村庄数据集 **发布方**：美国国务院人道主义信息股（已停用） · **数据来源**：[HDX](https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages) · **许可协议**：`other-pd-nr` · **最后更新时间**：2025-05-05 --- ## 摘要 本数据集记录了苏丹达尔富尔地区的村庄状况，涵盖美国政府确认的2003年2月至2010年12月期间遭受部分受损或完全损毁的村庄，同时也包含经证实未受任何损坏的村庄记录。 数据集中的每一行均代表一个带有地理定位的点位观测值。本数据集最后一次在HDX平台更新的时间为2025年5月5日。地理覆盖范围：**苏丹（SDN）**。 *本数据集已由[Electric Sheep Africa](https://huggingface.co/electricsheepafrica)整理为适用于机器学习的Parquet格式。* --- ## 数据集特征 | | | |---|---| | **领域** | 冲突与安全 | | **观测单元** | 带地理定位的点位观测值 | | **总样本行数** | 9183 | | **列数** | 14（9个数值型、5个分类型、0个日期时间型） | | **训练集拆分** | 7346行 | | **测试集拆分** | 1836行 | | **地理覆盖范围** | SDN | | **发布方** | 美国国务院人道主义信息股（已停用） | | **HDX最后更新时间** | 2025-05-05 --- ## 变量说明 **地理类变量** — `latitude`（取值范围：91624.0–14243299.0）、`lat_dd`（取值范围：9.2733–15.5889）、`longitude`（取值范围：215006.0–25534651.0）、`long_dd`（取值范围：21.835–27.39）、`yr_confirm`（取值范围：2003.0–2010.0）及另外4个变量。 **标识符与元数据类变量** — `name`（示例值：HASHABA、KUMA、HILLET IBRAHIM）、`esa_source`（取值：HDX）、`esa_processed`（更新时间：2026-04-05）。 **其他类变量** — `status`（取值：未受损坏（NO DAMAGE）、完全损毁（DESTROYED）、部分受损（DAMAGED））、`structures`（示例值：空、~25 of ~25、~50 of ~50）。 --- ## 快速上手 python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-darfur-damaged-and-destroyed-villages") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() --- ## 数据Schema | 列名 | 数据类型 | 缺失率 | 取值范围/示例值 | |---|---|---|---| | `name` | 字符型 | 68.8% | HASHABA、KUMA、HILLET IBRAHIM | | `latitude` | 浮点型 | 1.5% | 91624.0 – 14243299.0（均值：150722.5973） | | `lat_dd` | 浮点型 | 0.0% | 9.2733 – 15.5889（均值：12.8066） | | `longitude` | 浮点型 | 1.5% | 215006.0 – 25534651.0（均值：289943.1203） | | `long_dd` | 浮点型 | 0.0% | 21.835 – 27.39（均值：24.4526） | | `status` | 字符型 | 0.0% | 未受损坏（NO DAMAGE）、完全损毁（DESTROYED）、部分受损（DAMAGED） | | `structures` | 字符型 | 58.7% | 空、~25 of ~25、~50 of ~50 | | `yr_confirm` | 浮点型 | 71.8% | 2003.0 – 2010.0（均值：2004.7153） | | `aprox_str1` | 浮点型 | 78.1% | 5.0 – 1500.0（均值：105.8607） | | `aprox_str2` | 浮点型 | 70.5% | 10.0 – 4000.0（均值：117.8581） | | `yr_range1` | 浮点型 | 66.3% | 2003.0 – 2010.0（均值：2004.6276） | | `yr_range2` | 浮点型 | 66.3% | 2003.0 – 2010.0（均值：2004.7904） | | `esa_source` | 字符型 | 0.0% | HDX | | `esa_processed` | 字符型 | 0.0% | 2026-04-05 | --- ## 数值型变量统计摘要 | 列名 | 最小值 | 最大值 | 均值 | 中位数 | |---|---|---|---|---| | `latitude` | 91624.0 | 14243299.0 | 150722.5973 | 125656.0 | | `lat_dd` | 9.2733 | 15.5889 | 12.8066 | 12.9725 | | `longitude` | 215006.0 | 25534651.0 | 289943.1203 | 243237.0 | | `long_dd` | 21.835 | 27.39 | 24.4526 | 24.5558 | | `yr_confirm` | 2003.0 | 2010.0 | 2004.7153 | 2004.0 | | `aprox_str1` | 5.0 | 1500.0 | 105.8607 | 50.0 | | `aprox_str2` | 10.0 | 4000.0 | 117.8581 | 75.0 | | `yr_range1` | 2003.0 | 2010.0 | 2004.6276 | 2004.0 | | `yr_range2` | 2003.0 | 2010.0 | 2004.7904 | 2004.0 | --- ## 数据整理流程 原始数据通过CKAN API从HDX平台下载，并转换为Parquet格式。所有列名均转换为小写，并统一为蛇形命名法。常见的缺失值标记（`N/A`、`null`、`none`、`-`、`unknown`、`no data`、`#N/A`）被统一替换为`NaN`。移除了1列缺失率超过80%的列：`sec_town`。根据解析成功率（阈值为85%），将3列从字符串类型转换为数值型或日期时间型。本数据集使用固定随机种子（42）按照80/20的比例划分为训练集与测试集，并以Snappy压缩的Parquet格式存储。 --- ## 数据集局限性 - 本数据集源自美国国务院人道主义信息股（已停用），并未经Electric Sheep Africa独立验证。 - 自动化清洗流程无法修正原始数据集中的错报值、定义不一致或采样偏差问题。 - 以下列的缺失率超过20%，在建模过程中需谨慎使用：`name`、`structures`、`yr_confirm`、`aprox_str1`、`aprox_str2`、`yr_range1`、`yr_range2`。 - 请参阅[原始HDX数据集页面](https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages)以获取发布方提供的方法说明与注意事项。 --- ## 引用格式 bibtex @dataset{hdx_africa_darfur_damaged_and_destroyed_villages, title = {Darfur Damaged and Destroyed Villages}, author = {U.S. Department of State - Humanitarian Information Unit (inactive)}, year = {2025}, url = {https://data.humdata.org/dataset/darfur-damaged-and-destroyed-villages}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — 非洲机器学习数据集基础设施服务商，尼日利亚拉各斯。*
