electricsheepafrica/africa-historical-dry-spells-in-malawi

---
annotations_creators:
- no-annotation
language_creators:
- found
language:
- en
license: cc-by-4.0
multilinguality:
- monolingual
size_categories:
- 1K<n<10K
source_datasets:
- original
task_categories:
- tabular-regression
- other
task_ids: []
tags:
- africa
- humanitarian
- hdx
- electric-sheep-africa
- climate-weather
- mwi
pretty_name: "Historical dry spells in Malawi"
dataset_info:
splits:
- name: train
num_examples: 3084
- name: test
num_examples: 771
---

# Historical dry spells in Malawi

**Publisher:** HDX · **Source:** [HDX](https://data.humdata.org/dataset/historical-dry-spells-in-malawi) · **License:** `cc-by` · **Updated:** 2025-04-10

---

## Abstract

This dataset includes all the dry spells and rainy seasons in Malawi from 2000 till 2021 per admin2. In this dataset a dry spell is defined as 14 consecutive days with no more than 2 millimetres of cumulative rainfall. This dataset was produced as part of OCHA's Anticipatory Action pilot in Malawi.

Each row in this dataset represents time-series observations. Temporal coverage is indicated by the `dry_spell_first_date`, `dry_spell_last_date` column(s). Geographic scope: **MWI**.

*Curated into ML-ready Parquet format by [Electric Sheep Africa](https://huggingface.co/electricsheepafrica).*

---

## Dataset Characteristics

| | |
|---|---|
| **Domain** | Climate and environment |
| **Unit of observation** | Time-series observations |
| **Rows (total)** | 3,855 |
| **Columns** | 10 (3 numeric, 5 categorical, 2 datetime) |
| **Train split** | 3,084 rows |
| **Test split** | 771 rows |
| **Geographic scope** | MWI |
| **Publisher** | HDX |
| **HDX last updated** | 2025-04-10 |

---

## Variables

**Geographic** — `dry_spell_first_date`, `dry_spell_last_date`, `dry_spell_duration` (range 14.0–120.0), `dry_spell_rainfall` (range 0.0–6.6), `during_rainy_season` (range 0.0–1.0).

**Temporal** — `season_name` (not during a rainy season, 2007, 2004).

**Identifier / Metadata** — `pcode` (MW302, MW310, MW313), `esa_source` (HDX), `esa_processed` (2026-04-05).

**Other** — `adm2_en` (Machinga, Chikwawa, Neno).

---

## Quick Start

```python
from datasets import load_dataset

ds = load_dataset("electricsheepafrica/africa-historical-dry-spells-in-malawi")
train = ds["train"].to_pandas()
test = ds["test"].to_pandas()

print(train.shape)
train.head()
```

---

## Schema

| Column | Type | Null % | Range / Sample Values |
|---|---|---|---|
| `pcode` | object | 0.0% | MW302, MW310, MW313 |
| `adm2_en` | object | 0.0% | Machinga, Chikwawa, Neno |
| `season_name` | object | 26.7% | not during a rainy season, 2007, 2004 |
| `dry_spell_first_date` | datetime64[ns] | 0.0% | |
| `dry_spell_last_date` | datetime64[ns] | 0.0% | |
| `dry_spell_duration` | int64 | 0.0% | 14.0 – 120.0 (mean 33.1108) |
| `dry_spell_rainfall` | float64 | 0.0% | 0.0 – 6.6 (mean 1.2929) |
| `during_rainy_season` | float64 | 26.7% | 0.0 – 1.0 (mean 0.0159) |
| `esa_source` | object | 0.0% | HDX |
| `esa_processed` | object | 0.0% | 2026-04-05 |

---

## Numeric Summary

| Column | Min | Max | Mean | Median |
|---|---|---|---|---|
| `dry_spell_duration` | 14.0 | 120.0 | 33.1108 | 24.0 |
| `dry_spell_rainfall` | 0.0 | 6.6 | 1.2929 | 1.3 |
| `during_rainy_season` | 0.0 | 1.0 | 0.0159 | 0.0 |

---

## Curation

Raw data was downloaded from HDX via the CKAN API and converted to Parquet. Column names were lowercased and standardised to snake_case. Common missing-value markers (`N/A`, `null`, `none`, `-`, `unknown`, `no data`, `#N/A`) were unified to `NaN`. 2 column(s) were cast from string to numeric or datetime based on parse-success rate (>85% threshold). The dataset was split 80/20 into train and test partitions using a fixed random seed (42) and saved as Snappy-compressed Parquet.

---

## Limitations

- Data originates from HDX and has not been independently validated by ESA.
- Automated cleaning cannot correct for misreported values, definitional inconsistencies, or sampling bias in the original collection.
- The following columns have >20% missing values and should be treated with caution in modelling: `season_name`, `during_rainy_season`.
- Refer to the [original HDX dataset page](https://data.humdata.org/dataset/historical-dry-spells-in-malawi) for the publisher's own methodology notes and caveats.

---

## Citation

```bibtex
@dataset{hdx_africa_historical_dry_spells_in_malawi,
title = {Historical dry spells in Malawi},
author = {HDX},
year = {2025},
url = {https://data.humdata.org/dataset/historical-dry-spells-in-malawi},
note = {Repackaged for machine learning by Electric Sheep Africa (https://huggingface.co/electricsheepafrica)}
}
```

---

*[Electric Sheep Africa](https://huggingface.co/electricsheepafrica) — Africa's ML dataset infrastructure. Lagos, Nigeria.*