electricsheepafrica/africa-ethiopia-languages

--- annotations_creators: - no-annotation language_creators: - found language: - en license: cc-by-sa-4.0 multilinguality: - monolingual size_categories: - n<1K source_datasets: - original task_categories: - other task_ids: [] tags: - africa - humanitarian - hdx - electric-sheep-africa - languages - eth pretty_name: "Ethiopia: Languages" dataset_info: splits: - name: train num_examples: 57 - name: test num_examples: 14 --- # Ethiopia: Languages **Publisher:** CLEAR Global (previously Translators without Borders) · **Source:** [HDX](https://data.humdata.org/dataset/ethiopia-languages) · **License:** `cc-by-sa` · **Updated:** 2026-04-06 --- ## Abstract Data on languages spoken in Ethiopia, showing the main language spoken in the household by proportion of the population. Data is drawn from IPUMS International. For more resources on the languages of Ethiopia and language use in humanitarian contexts please visit: https://clearglobal.org/language-maps-and-data/ Each row in this dataset represents time-series observations. Temporal coverage is indicated by the `datetime_published`, `date_creation` column(s). Geographic scope: **ETH**. *Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).* --- ## Dataset Characteristics | | | |---|---| | **Domain** | Demographics and population | | **Unit of observation** | Time-series observations | | **Rows (total)** | 72 | | **Columns** | 16 (4 numeric, 10 categorical, 2 datetime) | | **Train split** | 57 rows | | **Test split** | 14 rows | | **Geographic scope** | ETH | | **Publisher** | CLEAR Global (previously Translators without Borders) | | **HDX last updated** | 2026-04-06 | --- ## Variables **Geographic** — `location_code` (ETH), `location_name` (Ethiopia), `location_level` (range 0.0–0.0), `reliability_score` (range 0.774–0.774), `representivity_rating` (very_high). **Temporal** — `datetime_published`, `date_creation`. **Demographic** — `language_code` (seba1251, awng1244, hara1271), `language_name` (Sebat Bet Gurage, Awngi, Harari), `language_rank` (range 1.0–72.0). **Outcome / Measurement** — `proportion_value` (range 0.0–0.3341). **Identifier / Metadata** — `dataset_name` (Ethiopia Census 2007 (IPUMS extract)), `source` (IPUMS International), `esa_source` (HDX), `esa_processed` (2026-04-07). **Other** — `url` (https://api.ipums.org/downloads/ipumsi/api/v1/extracts/2404719/ipumsi_00317.sav.gz). --- ## Quick Start ```python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-ethiopia-languages") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() ``` --- ## Schema | Column | Type | Null % | Range / Sample Values | |---|---|---|---| | `location_code` | object | 0.0% | ETH | | `location_name` | object | 0.0% | Ethiopia | | `location_level` | int64 | 0.0% | 0.0 – 0.0 (mean 0.0) | | `language_code` | object | 0.0% | seba1251, awng1244, hara1271 | | `language_name` | object | 0.0% | Sebat Bet Gurage, Awngi, Harari | | `language_rank` | int64 | 0.0% | 1.0 – 72.0 (mean 36.5) | | `proportion_value` | float64 | 0.0% | 0.0 – 0.3341 (mean 0.0139) | | `reliability_score` | float64 | 0.0% | 0.774 – 0.774 (mean 0.774) | | `dataset_name` | object | 0.0% | Ethiopia Census 2007 (IPUMS extract) | | `url` | object | 0.0% | https://api.ipums.org/downloads/ipumsi/api/v1/extracts/2404719/ipumsi_00317.sav.gz | | `source` | object | 0.0% | IPUMS International | | `datetime_published` | datetime64[ns] | 0.0% | | | `date_creation` | datetime64[ns] | 0.0% | | | `representivity_rating` | object | 0.0% | very_high | | `esa_source` | object | 0.0% | HDX | | `esa_processed` | object | 0.0% | 2026-04-07 | --- ## Numeric Summary | Column | Min | Max | Mean | Median | |---|---|---|---|---| | `location_level` | 0.0 | 0.0 | 0.0 | 0.0 | | `language_rank` | 1.0 | 72.0 | 36.5 | 36.5 | | `proportion_value` | 0.0 | 0.3341 | 0.0139 | 0.0007 | | `reliability_score` | 0.774 | 0.774 | 0.774 | 0.774 | --- ## Curation Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 2 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet. --- ## Limitations - Data originates from CLEAR Global (previously Translators without Borders) and has not been independently validated by ESA. - Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection. - Refer to the [original HDX dataset page](https://data.humdata.org/dataset/ethiopia-languages) for the publisher's own methodology notes and caveats. --- ## Citation ```bibtex @dataset{hdx_africa_ethiopia_languages, title = {Ethiopia: Languages}, author = {CLEAR Global (previously Translators without Borders)}, year = {2026}, url = {https://data.humdata.org/dataset/ethiopia-languages}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } ``` --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*annotations_creators: - 无注释 language_creators: - 公开获取 language: - 英语 license: CC BY-SA 4.0 multilinguality: - 单语种 size_categories: - 数据量少于1000条 source_datasets: - 原始数据集 task_categories: - 其他 task_ids: [] tags: - 非洲 - 人道主义 - HDX（人道主义数据交换平台） - Electric Sheep Africa - 语言 - ETH pretty_name: "埃塞俄比亚：语言" dataset_info: splits: - name: train num_examples: 57 - name: test num_examples: 14 # 埃塞俄比亚：语言 **发布方：** CLEAR Global（前身为Translators without Borders） · **数据源：** [HDX（人道主义数据交换平台）](https://data.humdata.org/dataset/ethiopia-languages) · **许可协议：** `cc-by-sa` · **更新时间：** 2026-04-06 --- ## 摘要 本数据集收录埃塞俄比亚境内的语言使用数据，展示按人口占比统计的家庭主要使用语言。数据源自IPUMS International（国际公共使用微数据系列国际版）。如需获取更多埃塞俄比亚语言及人道主义场景下语言使用的相关资源，请访问：https://clearglobal.org/language-maps-and-data/ 本数据集的每一行均代表时序观测数据，时间覆盖范围由`datetime_published`（数据发布时间）、`date_creation`（数据创建时间）字段标识。地理覆盖范围：**ETH**。 *本数据集已由[Electric Sheep Africa](https://huggingface.co/electricsheepafrica)整理为适配机器学习的Parquet格式。* --- ## 数据集特征 | | | |---|---| | **领域** | 人口与人口统计学 | | **观测单元** | 时序观测数据 | | **总数据行数** | 72 | | **字段总数** | 16（含4个数值型、10个分类型、2个日期时间型字段） | | **训练集划分** | 57条数据 | | **测试集划分** | 14条数据 | | **地理覆盖范围** | ETH | | **发布方** | CLEAR Global（前身为Translators without Borders） | | **HDX最后更新时间** | 2026-04-06 | --- ## 字段说明 **地理类字段** — `location_code`（国家代码，取值为ETH）、`location_name`（国家名称，值为Ethiopia）、`location_level`（取值范围0.0–0.0）、`reliability_score`（可信度评分，取值范围0.774–0.774）、`representivity_rating`（代表性评级，值为very_high）。 **时间类字段** — `datetime_published`、`date_creation`。 **人口统计类字段** — `language_code`（语言代码，取值为seba1251、awng1244、hara1271）、`language_name`（语言名称，对应Sebat Bet Gurage、Awngi、Harari）、`language_rank`（语言排名，取值范围1.0–72.0）。 **结果/测量类字段** — `proportion_value`（人口占比，取值范围0.0–0.3341）。 **标识/元数据类字段** — `dataset_name`（数据集名称，值为Ethiopia Census 2007 (IPUMS extract)）、`source`（数据源，值为IPUMS International）、`esa_source`（数据源，值为HDX）、`esa_processed`（处理时间，值为2026-04-07）。 **其他字段** — `url`（原始数据下载链接，值为https://api.ipums.org/downloads/ipumsi/api/v1/extracts/2404719/ipumsi_00317.sav.gz）。 --- ## 快速上手 python from datasets import load_dataset ds = load_dataset("electricsheepafrica/africa-ethiopia-languages") train = ds["train"].to_pandas() test = ds["test"].to_pandas() print(train.shape) train.head() --- ## 数据结构 | 字段名 | 数据类型 | 空值占比 | 取值范围/示例值 | |---|---|---|---| | `location_code` | 字符串（object） | 0.0% | ETH | | `location_name` | 字符串（object） | 0.0% | Ethiopia | | `location_level` | 整数型（int64） | 0.0% | 0.0 – 0.0 (mean 0.0) | | `language_code` | 字符串（object） | 0.0% | seba1251, awng1244, hara1271 | | `language_name` | 字符串（object） | 0.0% | Sebat Bet Gurage, Awngi, Harari | | `language_rank` | 整数型（int64） | 0.0% | 1.0 – 72.0 (mean 36.5) | | `proportion_value` | 浮点型（float64） | 0.0% | 0.0 – 0.3341 (mean 0.0139) | | `reliability_score` | 浮点型（float64） | 0.0% | 0.774 – 0.774 (mean 0.774) | | `dataset_name` | 字符串（object） | 0.0% | Ethiopia Census 2007 (IPUMS extract) | | `url` | 字符串（object） | 0.0% | https://api.ipums.org/downloads/ipumsi/api/v1/extracts/2404719/ipumsi_00317.sav.gz | | `source` | 字符串（object） | 0.0% | IPUMS International | | `datetime_published` | 日期时间型（datetime64[ns]） | 0.0% | 无 | | `date_creation` | 日期时间型（datetime64[ns]） | 0.0% | 无 | | `representivity_rating` | 字符串（object） | 0.0% | very_high | | `esa_source` | 字符串（object） | 0.0% | HDX | | `esa_processed` | 字符串（object） | 0.0% | 2026-04-07 | --- ## 数值型字段统计摘要 | 字段名 | 最小值 | 最大值 | 均值 | 中位数 | |---|---|---|---|---| | `location_level` | 0.0 | 0.0 | 0.0 | 0.0 | | `language_rank` | 1.0 | 72.0 | 36.5 | 36.5 | | `proportion_value` | 0.0 | 0.3341 | 0.0139 | 0.0007 | | `reliability_score` | 0.774 | 0.774 | 0.774 | 0.774 | --- ## 数据整理流程 原始数据通过CKAN API从HDX下载并转换为Parquet格式。字段名称统一转为小写，并采用蛇形命名法（snake_case）进行标准化。常见缺失值标记（如`N/A`、`null`、`none`、`-`、`unknown`、`no data`、`#N/A`）被统一替换为`NaN`。基于解析成功率（阈值>85%），将2个字段从字符串类型转换为数值型或日期时间型。本数据集以80:20的比例、固定随机种子（42）划分为训练集与测试集，并以Snappy压缩的Parquet格式存储。 --- ## 数据集局限性 - 原始数据源自CLEAR Global（前身为Translators without Borders），未经过Electric Sheep Africa（ESA）的独立验证。 - 自动化数据清洗无法修正原始数据收集中的错报值、定义不一致或抽样偏差问题。 - 如需了解发布方的方法说明与注意事项，请参阅[HDX原始数据集页面](https://data.humdata.org/dataset/ethiopia-languages)。 --- ## 引用格式 bibtex @dataset{hdx_africa_ethiopia_languages, title = {Ethiopia: Languages}, author = {CLEAR Global (previously Translators without Borders)}, year = {2026}, url = {https://data.humdata.org/dataset/ethiopia-languages}, note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)} } --- *[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — 非洲机器学习数据集基础设施。尼日利亚拉各斯。*