electricsheepafrica/africa-2018-malawi-ta-dataset-updated-admin3

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license: cc-by-4.0
multilinguality:
- monolingual
size_categories:
- 10K<n<100K
source_datasets:
- original
task_categories:
- other
task_ids: []
tags:
- africa
- humanitarian
- hdx
- electric-sheep-africa
- geodata
- mwi
pretty_name: "Malawi: 2018 Malawi Traditional Authority"
dataset_info:
splits:
- name: train
num_examples: 15017
- name: test
num_examples: 3754
---

# Malawi: 2018 Malawi Traditional Authority

**Publisher:** OCHA Regional Office for Southern and Eastern Africa (ROSEA) · **Source:** [HDX](https://data.humdata.org/dataset/2018_malawi_ta_dataset-updated-admin3) · **License:** `cc-by-igo` · **Updated:** 2025-04-08

---

## Abstract

2018 Malawi Traditional Authority ( Admin3 Border) dataset shared by National Statistical Office during Cyclone Idai Response, March 2019

Each row in this dataset represents subnational administrative unit observations. Data was last updated on HDX on 2025-04-08. Geographic scope: **MWI**.

*Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).*

---

## Dataset Characteristics

| | |
|---|---|
| **Domain** | Humanitarian and development data |
| **Unit of observation** | Subnational administrative unit observations |
| **Rows (total)** | 18,772 |
| **Columns** | 12 (7 numeric, 5 categorical, 0 datetime) |
| **Train split** | 15,017 rows |
| **Test split** | 3,754 rows |
| **Geographic scope** | MWI |
| **Publisher** | OCHA Regional Office for Southern and Eastern Africa (ROSEA) |
| **HDX last updated** | 2025-04-08 |

---

## Variables

**Geographic** — `region` (South, Central, North), `district` (Lilongwe, Mzimba, Mangochi).

**Demographic** — `total_pop` (range 1.0–999.0), `male` (range 1.0–999.0), `female` (range 1.0–999.0), `households` (range 1.0–944.0), `hh_size` (range 0.0–544.5).

**Identifier / Metadata** — `distcode` (range 101.0–315.0), `ta_name` (TA Chindi, TA Mtwalo, TA Msakambewa), `eacode` (range 10101001.0–31553902.0), `esa_source` (HDX), `esa_processed` (2026-04-05).

---

## Quick Start

```python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-2018-malawi-ta-dataset-updated-admin3")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()
```

---

## Schema

| Column | Type | Null % | Range / Sample Values |
|---|---|---|---|
| `region` | object | 0.0% | South, Central, North |
| `district` | object | 0.0% | Lilongwe, Mzimba, Mangochi |
| `distcode` | float64 | 0.0% | 101.0 – 315.0 (mean 230.7798) |
| `ta_name` | object | 0.0% | TA Chindi, TA Mtwalo, TA Msakambewa |
| `eacode` | int64 | 0.0% | 10101001.0 – 31553902.0 (mean 23087432.9938) |
| `total_pop` | float64 | 39.7% | 1.0 – 999.0 (mean 688.7133) |
| `male` | float64 | 2.0% | 1.0 – 999.0 (mean 447.705) |
| `female` | float64 | 2.5% | 1.0 – 999.0 (mean 472.8189) |
| `households` | float64 | 0.8% | 1.0 – 944.0 (mean 213.684) |
| `hh_size` | float64 | 0.8% | 0.0 – 544.5 (mean 4.489) |
| `esa_source` | object | 0.0% | HDX |
| `esa_processed` | object | 0.0% | 2026-04-05 |

---

## Numeric Summary

| Column | Min | Max | Mean | Median |
|---|---|---|---|---|
| `distcode` | 101.0 | 315.0 | 230.7798 | 209.0 |
| `eacode` | 10101001.0 | 31553902.0 | 23087432.9938 | 20904037.5 |
| `total_pop` | 1.0 | 999.0 | 688.7133 | 712.0 |
| `male` | 1.0 | 999.0 | 447.705 | 430.0 |
| `female` | 1.0 | 999.0 | 472.8189 | 457.0 |
| `households` | 1.0 | 944.0 | 213.684 | 204.0 |
| `hh_size` | 0.0 | 544.5 | 4.489 | 4.4 |

---

## Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 4 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

---

## Limitations

- Data originates from OCHA Regional Office for Southern and Eastern Africa (ROSEA) and has not been independently validated by ESA.
- Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
- The following columns have >20% missing values and should be treated with caution in modelling: `total_pop`.
- Refer to the [original HDX dataset page](https://data.humdata.org/dataset/2018_malawi_ta_dataset-updated-admin3) for the publisher's own methodology notes and caveats.

---

## Citation

```bibtex
@dataset{hdx_africa_2018_malawi_ta_dataset_updated_admin3,
title = {Malawi: 2018 Malawi Traditional Authority},
author = {OCHA Regional Office for Southern and Eastern Africa (ROSEA)},
year = {2025},
url = {https://data.humdata.org/dataset/2018_malawi_ta_dataset-updated-admin3},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}
```

---

*[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*