USGS-GlobalQuake

# USGS-GlobalQuake  USGS-GlobalQuake is a global earthquake catalog dataset built from the USGS earthquake event service. It provides a normalized JSONL catalog, an indexed SQLite database, direct database reading helpers, PyTorch dataloaders, and simple GRU training/inference examples. The dataset is designed for earthquake sequence modeling, global seismicity analysis, mainshock-aftershock sampling, and baseline forecasting experiments. Built with ❤️ for the seismology community. ## Dataset Summary - Source catalog: USGS earthquake event catalog. - Creator/maintainer: yuziye, [e-mail]. - Event count: 4,717,350. - Time coverage: `1900-01-05T19:00:00.000Z` to `2026-05-24T23:51:57.711Z`. - M6.0+ event count: 14,393. - Time zone: all timestamps ending with `Z` are UTC/GMT. - Release formats: - JSONL, one earthquake event per line. - SQLite, indexed for fast filtering and dataloading. - Python utility scripts for download, conversion, loading, training, and inference. ## Files ```text USGS-GlobalQuake/ README.md dataloader_example.py read_earthquake_events_example.py train_earthquake_catalog_demo.py infer_earthquake_catalog_gru.py figures/ usgs_globalquake_cover.png usgs_globalquake_cover.pdf data/ README.md usgs_global_all.jsonl usgs_global_all.sqlite scripts/ README.md fetch_global_earthquakes.py earthquake_csv_to_jsonl.py build_earthquake_db.py make_cover_figure.py utils/ __init__.py earthquake_catalog_loader.py gru_model.py ``` Data files: - `data/usgs_global_all.jsonl`: normalized event records, about 4.4 GB. - `data/usgs_global_all.sqlite`: indexed SQLite database, about 6.4 GB. Large data files are stored with Git LFS. ## Download ModelScope SDK: ```python from modelscope.msdatasets import MsDataset ds = MsDataset.load("cangyeone/USGS-GlobalQuake") ``` Git with LFS: ```bash git lfs install git clone https://www.modelscope.cn/datasets/cangyeone/USGS-GlobalQuake.git cd USGS-GlobalQuake ``` ## Quick Use Install runtime dependencies: ```bash pip install torch ``` Inspect one dataloader batch: ```bash python3 dataloader_example.py \ --db data/usgs_global_all.sqlite \ --mode fixed \ --start 2024-01-01T00:00:00Z \ --end 2024-01-08T00:00:00Z \ --max-events 64 ``` Read earthquake events directly from SQLite: ```bash python3 read_earthquake_events_example.py \ --db data/usgs_global_all.sqlite \ --start 2024-01-01T00:00:00Z \ --end 2024-01-02T00:00:00Z \ --min-magnitude 6 \ --limit 5 ``` Train a small GRU baseline: ```bash python3 train_earthquake_catalog_demo.py \ --db data/usgs_global_all.sqlite \ --mode fixed \ --start 2020-01-01T00:00:00Z \ --end 2020-02-01T00:00:00Z \ --window-days 1 \ --stride-days 1 \ --max-events 512 \ --epochs 1 ``` Run inference with a trained checkpoint: ```bash python3 infer_earthquake_catalog_gru.py \ --checkpoint checkpoints/gru_earthquake_demo.pt \ --db data/usgs_global_all.sqlite \ --mode fixed \ --start 2024-01-01T00:00:00Z \ --end 2024-01-08T00:00:00Z \ --limit 10 \ --output predictions.jsonl ``` ## Data Format Each JSONL row contains normalized fields and a lossless `raw` copy of the source row: ```json { "event_id": "us6000m0xl", "time": "2024-01-01T07:10:09.476Z", "time_epoch_ms": 1704093009476, "updated": "2026-01-20T05:59:43.803Z", "updated_epoch_ms": 1768888783803, "location": { "latitude": 37.4874, "longitude": 137.271, "depth_km": 10.0, "place": "2024 Noto Peninsula, Japan Earthquake" }, "magnitude": { "value": 7.5, "type": "mww", "error": 0.034, "station_count": 82 }, "quality": {}, "source": {}, "raw": {} } ``` SQLite table: `events` ```text event_id TEXT PRIMARY KEY time TEXT time_epoch_ms INTEGER updated TEXT updated_epoch_ms INTEGER latitude REAL longitude REAL depth_km REAL magnitude REAL magnitude_type TEXT place TEXT network TEXT status TEXT event_type TEXT raw_json TEXT ``` SQLite indexes: - `time_epoch_ms` - `(magnitude, time_epoch_ms)` - `(latitude, longitude)` ## Dataloader Import: ```python from torch.utils.data import DataLoader from utils.earthquake_catalog_loader import ( EarthquakeCatalogArgs, FixedWindowEarthquakeDataset, MainAftershockEarthquakeDataset, earthquake_collate_fn, ) ``` Fixed interval sampling: ```python args = EarthquakeCatalogArgs( db_path="data/usgs_global_all.sqlite", max_events=512, fixed_window_days=1.0, fixed_stride_days=1.0, ) dataset = FixedWindowEarthquakeDataset( args, start="2020-01-01T00:00:00Z", end="2021-01-01T00:00:00Z", ) loader = DataLoader(dataset, batch_size=4, collate_fn=earthquake_collate_fn) ``` Mainshock-aftershock sampling: ```python args = EarthquakeCatalogArgs( db_path="data/usgs_global_all.sqlite", max_events=512, main_min_magnitude=6.0, pre_days=1.0, post_days=7.0, radius_km=300.0, ) dataset = MainAftershockEarthquakeDataset(args) loader = DataLoader(dataset, batch_size=2, collate_fn=earthquake_collate_fn) ``` Default mainshock-aftershock settings: - Mainshock threshold: `M >= 6.0`. - Time window: 1 day before mainshock to 7 days after mainshock. - Epicentral radius: 300 km. The 300 km default is a practical first-pass value for M6+ sequences. For M7+ events, `500 km` can be more inclusive. For dense regional catalogs, `100-200 km` can reduce unrelated background seismicity. ## Direct Database Reads The loader also provides direct SQLite helpers without constructing a Dataset: ```python from utils.earthquake_catalog_loader import ( read_earthquake_events, read_earthquake_events_near, read_event_by_id, ) events = read_earthquake_events( "data/usgs_global_all.sqlite", start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z", min_magnitude=6.0, limit=10, ) nearby = read_earthquake_events_near( "data/usgs_global_all.sqlite", center_latitude=37.4874, center_longitude=137.271, radius_km=300.0, start="2024-01-01T00:00:00Z", end="2024-01-02T00:00:00Z", ) ``` ## Batch Tensor Format The collated PyTorch batch contains: ```text wave: [B, S, 1, 8] pos: [B, S] geo: [B, S, 18] valid_mask: [B, S] token_type: [B, S] magnitude: [B, S] magnitude_mw: [B, S] magnitude_raw: [B, S] events: Python metadata list ``` `S` is `max_events`. Padding tokens have `valid_mask=False`. Each earthquake event is represented as one sequence element: - `pos`: relative time in days from the sample window start. - `wave`: compact event-feature tensor with estimated Mw, depth, normalized lon/lat, normalized Earth-centered xyz, and relative time. - `geo`: normalized lon/lat/depth, normalized Earth-centered xyz, estimated Mw, relative time, and mainshock-relative features. Coordinates in metadata: - `latlon_depth`: latitude, longitude, depth in km. - `xyz_earth_center_km`: Earth-centered Cartesian coordinates in km, with Earth center as `(0, 0, 0)` and radius `6371 - depth_km`. ## Magnitude Homogenization The SQLite database preserves the original USGS `magnitude` and `magnitude_type`. The dataloader converts magnitudes to an estimated moment magnitude scale by default: ```python EarthquakeCatalogArgs( db_path="data/usgs_global_all.sqlite", use_mw_magnitude=True, magnitude_filter_uses_mw=True, ) ``` Default conversion rules: ```text Mw-like types: Mw = M mb: Mw = 0.85 * mb + 1.03 valid around 3.5 <= mb <= 6.2 Ms low range: Mw = 0.67 * Ms + 2.07 valid around 3.0 <= Ms <= 6.1 Ms high range: Mw = 0.99 * Ms + 0.08 valid around 6.2 <= Ms <= 8.2 ML: Mw = 0.603 * ML + 2.072 used only around 2.7 <= ML <= 6.0 MD: Mw = 0.764 * MD + 1.379 used only around 3.7 <= MD <= 6.0 unknown: Mw = M, flagged as identity_unknown_mag_type ``` The `mb` and `Ms` rules follow common Scordilis-style global conversion relations. The `ML` and `MD` rules are more regional and are intentionally conservative: outside their nominal ranges the loader keeps the original value unless `convert_magnitude_outside_valid_range=True`. ## Rebuilding the Dataset Download CSV from USGS: ```bash python3 scripts/fetch_global_earthquakes.py \ --start 1900-01-01 \ --end 2026-05-25 \ --output data/usgs_global_all.csv \ --parts-dir data/parts ``` Convert CSV to JSONL: ```bash python3 scripts/earthquake_csv_to_jsonl.py \ --input data/usgs_global_all.csv \ --output data/usgs_global_all.jsonl ``` Build SQLite: ```bash python3 scripts/build_earthquake_db.py \ --input data/usgs_global_all.jsonl \ --output data/usgs_global_all.sqlite ``` ## Intended Uses This dataset can be used for: - earthquake sequence modeling; - mainshock-aftershock context construction; - global seismicity statistics; - baseline magnitude forecasting experiments; - geospatial and temporal representation learning. ## Limitations - The catalog is not a complete record of every earthquake on Earth. Global detection completeness varies by time, region, magnitude, network density, and source catalog practice. - Small events are much less complete globally than large events. - Magnitude homogenization is empirical. The provided Mw conversion is useful for modeling convenience but should not replace authoritative seismological review. - The GRU script is a baseline example, not a production earthquake prediction system. ## Source and Attribution The event data are derived from the USGS earthquake event service: - USGS Earthquake Hazards Program: https://earthquake.usgs.gov/ - USGS FDSN Event Web Service: https://earthquake.usgs.gov/fdsnws/event/1/ Please cite or acknowledge USGS when using the underlying earthquake catalog. ## License and Terms This repository contains scripts and a processed dataset derived from USGS earthquake catalog data. USGS data are generally public domain in the United States, but users should review USGS terms, attribution guidance, and any applicable local requirements before redistribution or publication. The provided scripts are released for research and educational use. No warranty is provided. Built with ❤️ for the seismology community.