Turkmenistan methane point source detections and retrievals from Landsat 5 (1986-2011)

# Turkmenistan methane point source detections and retrievals from Landsat 5 (1986-2011) This dataset contains replication data for our paper entitled "[Space-borne assessment of the Soviet Union collapse on the methane slowdown](https://doi.org/10.31223/X5G67G)". ## Usage notes To open and visualize the data files in `augmented_labelled_data.tar.gz`, a Python environment with the NumPy and PyTorch libraries is required. Files archived in `retrieval_and_masks.tar.gz` could be opened and visualized using [NASA's Panoply data viewer](https://www.giss.nasa.gov/tools/panoply/). ## Description of the data and file structure 1\) `augmented_labelled_data.tar.gz` contains human-labelled plume masks and the corresponding Landsat 5 measurements. * The package includes both positive (with plume) and negative (without plume) satellite scenes. The methane plumes were labeled by the two leading authors of this dataset. Four folders are archived: * 'neg_labeller_1' contains **2326** negative satellite scenes labelled by labeler 1 in '.npz' format. * 'neg_labeller_2' contains **2505** negative satellite scenes labelled by labeler 2 in '.npz' format. * 'pos_labeller_1' contains **1864** positive satellite scenes labelled by labeler 1 in '.npz' format. * 'pos_labeller_2' contains **1449** positive satellite scenes labelled by labeler 2 in '.npz' format. * The name convention for '.npz' files is "*Satellite_Longitude_Latitude_Date_Localtime_MonthOverpassNum_AugmentID.npz*". Satellites include Landsat 7 (*L7*), Landsat 8 (*L8*), and Sentinel-2 (*S2*). MonthOverpassNum stands for the number of satellite overpass for this location in the month. Filenames with an AugmentID are datasets augmented following the method described in the paper. Filenames without an AugmentID are raw data sets without any augmentation. * The '.npz' files could be loaded using the NumPy Python library use these commands: ``` import numpy as np npzfile = np.load(‘file.npz’) npzfile.files ``` The commands will return a list of the NumPy arrays, which can be individually opened by: ``` data = npzfile['variable'][:] ``` * Four variables are archived in each '.npz' file: '*pred*', '*mask*', '*lon*', and '*lat*'. * The '*pred*' variable contains 13 128x128 images sampled by the satellites. The order of the 13 images is the *dR* channel, the red channel, the green channel, the blue channel, the near infrared (*NIR*) channel, the first shortwave infrared (*SWIR*) channel, the second *SWIR* channel, the estimated cloud percent, the background (*bg*) scene for *dR*, the grayscale RGB, the normalized difference vegetation index (*NDVI*), and two difference fields between *dR* and *dRbg*. More details could be found in the paper mentioned above. * The '*mask*' variable is 128x128 binary images, where "1" stands for plume and "0" stands for no plume. * '*lon*' and '*lat*' stand for the longitude and latitude information, respectively. * All variables are NumPy arrays. 2\) `retrieval_and_masks.tar.gz` contains the methane retrievals and the predicted methane plume masks from the Landsat 5 satellite. * One folder is archived: * 'retrieval_and_masks' contains **688** '.nc' files for the retrieved methane columns and the predicted plume masks. * Filename convention is "*Date_Longitude_Latitude.nc*". The '.nc' files are netCDF4 files that could be directly opened and visualized using [NASA's Panoply data viewer](https://www.giss.nasa.gov/tools/panoply/). * Four variables are listed: * the retrieved CH4 columns (variable name '*ch4*') in units of '*mol/m2*', * the estimated background CH4 columns (variable name '*ch4bg*') in units of '*mol/m2*', * the binary plume masks (variable name '*plume_mask*') predicted by the deep learning models, * and the date and time information (variable name '*time*') as days since 1950-01-01. 3\) `L5_detections.csv` is a comma separated list file containing the methane point source detected over Turkmenistan from Landsat 5. The file could be opened and viewed directly using Excel, LibreOffice Calc, etc. Reported are location, date and time, estimated flux rates, and the uncertainties of flux rates associated with the detections. * Five variables are listed: *Longitude*, *Latitude*, *Datetime*, *Q* and *dQ*. * Datetime is in format of MM/DD/YYYY. * *Q* is the estimated flux rates of the detected methane point sources, reported in units of '*kg/hr*'. * *dQ* is the uncertainties associated with the estimated flux rates, reported in units of '*kg/hr*'. ## Sharing/Access information The top-of-atmosphere (TOA) reflectance measurements from Landsat 5, Landsat 8, and Sentinel-2 are processed from Google Earth Engine (GEE, ). The codebase to reproduce results in the paper is available at . The methane plume mask annotation tool is available at .