juliensimon/substorm-onsets

---
license: cc-by-4.0
pretty_name: "Substorm Onset Events (SuperMAG)"
language:
- en
description: "Magnetospheric substorm onset events from 5 detection algorithms via SuperMAG (253,319 events, 1975-2024)."
task_categories:
- tabular-classification
- time-series-forecasting
tags:
- space
- space-weather
- substorm
- magnetosphere
- aurora
- geomagnetic
- supermag
- open-data
- tabular-data
- parquet
size_categories:
- 100K<n<1M
configs:
- config_name: default
data_files:
- split: train
path: data/substorm_onsets.parquet
default: true
---

# Substorm Onset Events (SuperMAG)

<div align="center">
<img src="banner.jpg" alt="Aurora borealis blankets the Earth, seen from the ISS" width="400">
<p><em>Credit: NASA</em></p>
</div>

*Part of the [Space Weather Datasets](https://huggingface.co/collections/juliensimon/space-weather-datasets-69c24cae98f1666f2101ca70) collection on Hugging Face.*

A consolidated catalog of **253,319** magnetospheric substorm onset events spanning **1975--2024**, combining five independent detection algorithms from the [SuperMAG](https://supermag.jhuapl.edu/) collaboration. This is the most comprehensive substorm event list available, enabling multi-algorithm comparison and consensus studies.

## Dataset description

Magnetospheric substorms are fundamental space weather events driven by the solar wind's interaction with Earth's magnetic field. During a substorm, magnetic energy stored in the magnetotail is explosively released, accelerating charged particles that stream along field lines into the polar regions. This produces sudden auroral brightenings — the dramatic intensification of the Northern and Southern Lights — along with rapid changes in ground-level magnetic fields detected by magnetometer networks worldwide.

This dataset merges five complementary onset detection methods:

| Source | Events | Detection method |
|--------|-------:|-----------------|
| `forsyth` | 120,069 | SML/SMU expansion-recovery (ground magnetometers) |
| `newell` | 81,914 | SML index (ground magnetometers) |
| `ohtani` | 44,606 | SML bay detection (ground magnetometers) |
| `frey` | 4,191 | IMAGE/FUV auroral imaging (space-based) |
| `liou` | 2,539 | Polar UVI auroral imaging (space-based) |

**Ground-based methods** (246,589 events) detect substorms through characteristic negative bays in the SML (SuperMAG Lower) index — a measure of the westward auroral electrojet current. **Space-based methods** (6,730 events) directly observe the initial auroral brightening using ultraviolet imagers aboard the IMAGE and Polar satellites.

Each algorithm has different sensitivity and false-positive rates, so researchers often require onset confirmation across multiple lists. The `source` column enables filtering by algorithm or finding consensus events where multiple methods agree within a time window.

## Schema

| Column | Type | Description |
|--------|------|-------------|
| `datetime_utc` | datetime | Substorm onset time (UTC) |
| `mlt_hours` | float64 | Magnetic Local Time of onset (hours, 0-24) |
| `magnetic_latitude_deg` | float64 | Magnetic latitude of onset (degrees) |
| `geographic_longitude_deg` | float64 | Geographic longitude of onset (degrees) |
| `geographic_latitude_deg` | float64 | Geographic latitude of onset (degrees) |
| `source` | string | Detection algorithm: newell, forsyth, ohtani, frey, liou |
| `method` | string | Detection method description |

## Quick stats

- **253,319** total substorm onset events
- **1975--2024** temporal coverage
- **5** independent detection algorithms
- **246,589** ground magnetometer detections, **6,730** auroral imaging detections

## Usage

```python
from datasets import load_dataset

ds = load_dataset("juliensimon/substorm-onsets", split="train")
df = ds.to_pandas()

# Events per algorithm
print(df["source"].value_counts())

# Annual substorm rate by algorithm
import matplotlib.pyplot as plt
df["year"] = df["datetime_utc"].dt.year
df.groupby(["year", "source"]).size().unstack().plot(figsize=(12, 5))
plt.ylabel("Substorm onsets per year")
plt.title("Annual Substorm Rate by Detection Algorithm")
plt.show()

# MLT distribution — substorms peak near midnight
df["mlt_hours"].hist(bins=48, alpha=0.7)
plt.xlabel("Magnetic Local Time (hours)")
plt.ylabel("Count")
plt.title("Substorm Onset MLT Distribution")
plt.show()

# Find consensus events (multiple algorithms within 10 minutes)
from datetime import timedelta
newell = df[df["source"] == "newell"]["datetime_utc"]
ohtani = df[df["source"] == "ohtani"]["datetime_utc"]
```

## Data source

SuperMAG substorm onset lists, provided by the Johns Hopkins University Applied Physics Laboratory:
- Newell & Gjerloev (2011), [doi:10.1029/2010JA016141](https://doi.org/10.1029/2010JA016141)
- Forsyth et al. (2015), [doi:10.1002/2015JA021343](https://doi.org/10.1002/2015JA021343)
- Ohtani & Gjerloev (2020), [doi:10.1029/2019JA027680](https://doi.org/10.1029/2019JA027680)
- Frey et al. (2004), [doi:10.1029/2003JA010300](https://doi.org/10.1029/2003JA010300)
- Liou (2010), [doi:10.1016/j.jastp.2009.08.005](https://doi.org/10.1016/j.jastp.2009.08.005)

## Related datasets

- [Dst Index](https://huggingface.co/datasets/juliensimon/dst-index) — Hourly geomagnetic storm index
- [Kp Index](https://huggingface.co/datasets/juliensimon/kp-index) — 3-hourly geomagnetic activity
- [AE Index](https://huggingface.co/datasets/juliensimon/ae-index) — Auroral electrojet indices
- [DONKI Space Weather](https://huggingface.co/datasets/juliensimon/donki) — CMEs, flares, and geomagnetic storms

## Pipeline

Source code: [juliensimon/space-datasets](https://github.com/juliensimon/space-datasets)

## Support

If you find this dataset useful, please give it a ❤️ on the [dataset page](https://huggingface.co/datasets/juliensimon/substorm-onsets) and share feedback in the Community tab! Also consider giving a ⭐ to the [space-datasets](https://github.com/juliensimon/space-datasets) repo.

## Citation

```bibtex
@dataset{substorm_onsets,
author = {Simon, Julien},
title = {Substorm Onset Events (SuperMAG)},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/datasets/juliensimon/substorm-onsets},
note = {Consolidated from SuperMAG: Newell, Forsyth, Ohtani, Frey, Liou lists}
}
```

## License

[CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)