A Global Multi-Decadal Convection Tracking Database from ISCCP-H (1983-2017)

A Global Multi-Decadal Convection Tracking Database from ISCCP-H (1983-2017) ISCCP-H CT: a global, 34-year (July 1983 - June 2017) convection-tracking database derived from International Satellite Cloud Climatology Project H-series (ISCCP-H) infrared observations, produced with the Tracking and Object-Based Analysis of Clouds (tobac) framework. This dataset accompanies the manuscript: Luo, Z. J., Wang, L.-P., Selevich, Y., Takahashi, H., Wu, C.-L., Jhang, H., Lin, S.-C., van den Heever, S. C., Machado, L. A., Rossow, W., and Freeman, S. (2026). A Global Multi-Decadal Convection Tracking Database from ISCCP-H: Dataset Description and Convective Lifecycle Analysis. Earth and Space Science (in review, MS 2026EA005442-T). Description Convective cloud systems were detected and tracked in ISCCP-H infrared brightness temperature fields (3-hourly temporal resolution, 10-km spatial resolution) using the tobac framework. Detection used TB minima with dual thresholds of 245 K and 220 K (weighted_diff positioning), segmentation at 245 K, and predictive linking with a maximum velocity of 30 m/s (see the accompanying manuscript Methods for the full tracking configuration). Each record in the dataset is one observation of one convective system (feature) at one 3-hourly time step, linked into cell families by the tracker so that the full Lagrangian lifecycle of each system can be reconstructed. For every tracked system the dataset provides: position and elliptical geometry, brightness temperature statistics (minimum, mean, maximum, standard deviation, and percentiles), size measures (pixel counts, equivalent radius, deep-convective pixel fractions), lifecycle timing (time since initiation, total lifetime), tracking-quality diagnostics (overlap percentages), and environmental context (surface type, wind speed and direction, cloud optical thickness). The number of detected convective systems remains approximately stable over the 34-year record, while their organization, intensity, and lifecycles vary substantially. The database supports composite lifecycle analysis, lifecycle-type classification, and long-term climatology of convective systems, as described in the accompanying manuscript. Coverage and Volume Property Value Temporal coverage July 1983 - June 2017 (34-year record) Temporal resolution 3-hourly Spatial coverage Global, 60S - 60N, 0 - 360E Source spatial resolution 10 km (ISCCP-H infrared) Total records 67,257,982 system-level 3-hourly observations Columns 57 documented variables (plus one legacy index column, see Known Limitations) Format Apache Parquet, one file per calendar year, Snappy compression Total size approximately 11.6 GB (35 files) Metadata convention CF-1.8 attributes embedded in each Parquet file schema Note: 1983 contains July - December only and 2017 contains January - June only; all other years are complete. Files One Parquet file per calendar year, named ISCCP-H_CT_<year>.parquet (35 files, 171 - 376 MB each). A checksums.txt (MD5) and manifest.csv (per-file row counts and sizes) are included. Each file contains a year column, so any subset of files can be combined and filtered directly. File Rows Size (MB) ISCCP-H_CT_1983.parquet 972,548 171.2 ISCCP-H_CT_1984.parquet 1,849,779 325.7 ISCCP-H_CT_1985.parquet 1,848,539 326.7 ISCCP-H_CT_1986.parquet 1,885,680 332.8 ISCCP-H_CT_1987.parquet 1,984,137 347.9 ISCCP-H_CT_1988.parquet 1,902,938 334.1 ISCCP-H_CT_1989.parquet 1,870,502 329.6 ISCCP-H_CT_1990.parquet 1,895,297 333.8 ISCCP-H_CT_1991.parquet 1,913,758 336.5 ISCCP-H_CT_1992.parquet 2,009,737 351.9 ISCCP-H_CT_1993.parquet 2,038,530 357.3 ISCCP-H_CT_1994.parquet 1,981,666 348.8 ISCCP-H_CT_1995.parquet 1,854,447 324.5 ISCCP-H_CT_1996.parquet 1,896,963 331.4 ISCCP-H_CT_1997.parquet 1,870,999 327.0 ISCCP-H_CT_1998.parquet 1,846,262 323.3 ISCCP-H_CT_1999.parquet 1,956,837 341.4 ISCCP-H_CT_2000.parquet 1,964,305 342.9 ISCCP-H_CT_2001.parquet 1,968,888 344.2 ISCCP-H_CT_2002.parquet 2,044,097 356.1 ISCCP-H_CT_2003.parquet 2,028,431 354.5 ISCCP-H_CT_2004.parquet 2,048,417 358.0 ISCCP-H_CT_2005.parquet 2,031,521 354.6 ISCCP-H_CT_2006.parquet 2,122,503 372.0 ISCCP-H_CT_2007.parquet 2,086,893 364.5 ISCCP-H_CT_2008.parquet 2,140,794 375.7 ISCCP-H_CT_2009.parquet 2,135,005 374.3 ISCCP-H_CT_2010.parquet 2,104,052 367.8 ISCCP-H_CT_2011.parquet 2,082,685 363.7 ISCCP-H_CT_2012.parquet 2,006,012 349.8 ISCCP-H_CT_2013.parquet 2,008,039 350.0 ISCCP-H_CT_2014.parquet 1,856,287 325.5 ISCCP-H_CT_2015.parquet 1,970,479 343.6 ISCCP-H_CT_2016.parquet 2,038,904 354.4 ISCCP-H_CT_2017.parquet 1,042,051 181.1 Data Schema Identifiers Column Description Units frame Time frame ID (resets each year) 1 feature Convective feature ID (resets each year) 1 cell Convective cell family ID (resets each year) 1 global_frame_id Globally unique frame ID, format YYYY_frameid 1 global_feature_id Globally unique feature ID, format YYYY_featureid 1 global_cell_id Globally unique cell family ID, format YYYY_cellid 1 Time Column Description Units datetime UTC timestamp of observation seconds since 1970-01-01 year / month / day / time Calendar components of the timestamp 1 time_cell Elapsed time since cell initiation seconds total_hours Hours since cell initiation hours lifetime_hours Total cell lifetime duration hours lifetime_num_cs Number of convective systems in the cell family 1 Location and Geometry Column Description Units latitude / longitude Centroid of convective system (tobac weighted_diff) degrees north / east central_latitude / central_longitude Fitted ellipse center degrees north / east min_lat / max_lat / min_lon / max_lon Bounding extent degrees radius Equivalent circular radius km max_radius_cell Maximum radius within the cell family km semi_major / semi_minor Fitted ellipse axes km eccentricity Ellipse eccentricity (0 = circular, 1 = linear) 1 inclination Ellipse major axis inclination degrees Brightness Temperature Column Description Units minTB_feature / minTB_cell Minimum TB of system / cell family K avgTB_feature / avgTB_cell Mean TB of system / cell family K maxTB_feature / maxTB_cell Maximum TB of system / cell family K std_dev_tb Standard deviation of TB K 10th, 25th, 50th, 75th, 90th, 95th, 99th TB percentiles within the system K cs_gradient Cloud system temperature gradient K per km Size and Convective Intensity Column Description Units pixel_count Number of pixels in the system 1 pixels_below_220 Pixels with TB below 220 K (deep convection) 1 pixels_below_200 Pixels with TB below 200 K (extreme convection) 1 convective_fraction Fraction of deep convective pixels percent Tracking Quality Column Description Units percent_overlap Overlap with previous time step (higher = better tracking) percent percent_non_overlap Non-overlapping area percentage percent squared_corr Squared correlation coefficient 1 Environment Column Description Units land_water_mask Surface type, "land" or "water" (pixel majority) flag wind_speed Environmental wind speed m per s wind_dir Wind direction (degrees from north) degrees wind_dir_letter 16-point compass wind direction (N, NNE, ..., NNW) flag avg_optical_thickness / max_optical_thickness Cloud optical thickness 1 Missing or invalid values are stored as IEEE 754 NaN; no imputation is performed. Full CF-1.8 attributes (long_name, units, valid ranges, fill values) are embedded in each Parquet file and can be inspected with PyArrow. Usage Python with Polars (lazy scan, memory efficient): ```python import polars as pl df = ( pl.scan_parquet("ISCCP-H_CT_*.parquet") .filter( (pl.col("year").is_between(2010, 2016)) & (pl.col("minTB_feature") < 220) & (pl.col("percent_overlap") > 50) ) .select(["global_cell_id", "datetime", "latitude", "longitude", "minTB_feature", "radius", "lifetime_hours"]) .collect() ) ``` Python with pandas (single year): ```python import pandas as pd df = pd.read_parquet("ISCCP-H_CT_2016.parquet") ``` DuckDB (SQL over all years): sql SELECT year, COUNT(*) AS n_obs, COUNT(DISTINCT global_cell_id) AS n_cells FROM 'ISCCP-H_CT_*.parquet' GROUP BY year ORDER BY year; Reading embedded CF-1.8 metadata: ```python import pyarrow.parquet as pq schema = pq.read_schema("ISCCP-H_CT_2016.parquet") print(schema.field("minTB_feature").metadata) ``` Recommended Quality Control Suggested baseline filters for physical analysis (adjust to application): percent_overlap > 50 (reliable tracking) pixel_count > 10 (exclude marginal detections) minTB_feature between 180 and 300 K (physically valid range) Known Limitations frame, feature, and cell IDs reset every year. Use global_frame_id, global_feature_id, and global_cell_id for multi-year work. Cell families are not tracked across the year boundary; systems alive at 31 December receive new cell IDs in January. 1983 covers July - December only; 2017 covers January - June only. Each file contains a __index_level_0__ column, a legacy pandas index retained for reproducibility with the manuscript analysis. It carries no physical meaning and should be ignored. High-latitude coverage is limited to 60S - 60N by the ISCCP-H input data. The flag_values attribute embedded in the files for wind_dir_letter lists 8 principal directions, but the data contain all 16 compass points (N, NNE, ..., NNW). Geostationary satellite transitions within the ISCCP-H record may introduce inhomogeneities; see the accompanying manuscript for the homogeneity assessment. Provenance and Related Resources Source data: ISCCP H-series Climate Data Record (NOAA NCEI), https://doi.org/10.7289/V5QZ281S ; described by Young et al. (2018), https://doi.org/10.5194/essd-10-583-2018 Tracking software: tobac (Tracking and Object-Based Analysis of Clouds), Heikenfeld et al. (2019), https://doi.org/10.5194/gmd-12-4159-2019 ; tobac v1.5, Sokolowsky et al. (2024), https://doi.org/10.5194/gmd-17-5309-2024 (see the accompanying manuscript for the version used) Companion resource: AUX-GEOIR higher-resolution tracking database in development for the NASA INCUS mission License Creative Commons Attribution 4.0 International (CC BY 4.0). Citation Please cite both the dataset and the accompanying manuscript: Dataset: Luo, Z. J., et al. (2026). A Global Multi-Decadal Convection Tracking Database from ISCCP-H (1983-2017) [Dataset]. Zenodo. https://doi.org/10.5281/zenodo.21505419 Paper: Luo, Z. J., et al. (2026). A Global Multi-Decadal Convection Tracking Database from ISCCP-H: Dataset Description and Convective Lifecycle Analysis. Earth and Space Science (in review). Funding Funding information is provided in the acknowledgments of the accompanying manuscript. Contact Corresponding author: Zhengzhao Johnny Luo, The City University of New York ([e-mail]) Version 1.0.0 (initial public release, corresponds to the data version analyzed in MS 2026EA005442-T)