Code and Data for Statistical Decomposition and Machine Learning to Clean In-Situ Spaceflight Magnetic Field Measurements

This repository contains all of the MATLAB code and data required to regenerate the major contents of the manuscript named above. Note that FOUR major subdirectories are contained at the top level of the repository:<br>1. _libThe *_lib* subdirectory contains all of the generic functions used throughout. * <strong><em>calcSpectrum.m</em></strong> returns the power and amplitude spectrums for a given input using one of several filter types. * <strong><em>mssa.m</em></strong> returns the sub-signals output by Multichannel Singular Spectrum Analysis for a pair of inputs with a given window length. * <strong><em>ssa.m</em></strong> returns the sub-signals outpuit by Singular Spectrum Analysis for a single input with a given window length. * <strong><em>uniform_detrend.m</em></strong> returns the output of a uniform filter with given length applied to the input.<br>2. _sourceThe *_source* subdirectory contains the source data used for analysis of the proposed method. Specifically, Common Data Format (.cdf) files are stored from both the Parker Solar Probe FIELDS magnetometers (MAGo and MAGi) and the CASSIOPE e-POP/Swarm-Echo Magnetic Field Instrument (MGF).Note that the Parker Solar Probe FIELDS instrument suite was designed, built, and is operated by a team lead by the Space Sciences Laboratory at the University of California, Berkeley (PI: S.D. Bale). MAGo measurements from PSP are publicly available at https://fields.ssl.berkeley.edu/data/. The MAGi data stored here was generated for this manuscript.The CASSIOPE e-POP MGF data stored here (inboard and outboard) is publicly available through https://epop.phys.ucalgary.ca/data-handbook/getting-e-pop-data/.<br>3. ePOP_MGFThe <em>ePOP_MGF</em> subdirectory contains all of the analysis code required to regenerate the ePOP figures and analysis presented in the manuscript. * <strong><em>epop_dataset_gen.m</em></strong> reads in 5 days worth of ePOP magnetic field data from the *_source* folder, chops the measurements into 40-second intervals, and omits any intervals that contain NaN values. The resulting data is stored in <em>.mat</em> format in the *_intermediates* subdirectory. * <strong><em>component_dataset_gen.m</em></strong> takes the intermediate data generated from <em>epop_dataset_gen</em> and passes each interval (for inboard and outboard magnetometers) through the MSSA function and associated labeling scheme. The sub-signals generated by MSSA are saved in <em>.mat</em> format in the *_intermediates* subdirectory. * <strong><em>train_neural_net.m</em></strong> reads in the output of the <em>component_datset_gen</em> and uses the data to train the neural network detailed in the manuscript. The trained model is stored in the *_intermediates* subdirectory. * <strong><em>'</em>_data_gen.m'</strong>* files read in the appropriate CDF file and convert only certain time intervals of the relevant magnetic field data to <em>.mat</em> format for use in figure generation and analysis. * *<strong>'<em>_fig_gen.m'</em></strong> files use the trained neural network and data generated by *_data_gen* to generate the appropriate figures and numerical analysis. These figures are stored in the *_outputs* subdirectory.<br>4. PSP_MAGThe <em>PSP_MAG</em> subdirectory contains all of the analysis code required to regenerate the PSP figures and analysis presented in the manuscript. * <strong><em>psp_dataset_gen.m</em></strong> reads in the PSP magnetic field data from the *_source* folder. The resulting data is stored in <em>.mat</em> format in the *_intermediates* subdirectory. Note that in order to read the PSP CDF files, the NASA SPDF CDF MATLAB patch is required. This patch is documented at: https://cdf.gsfc.nasa.gov/html/matlab_cdf_patch.html. Note that the 'addpath()' MATLAB command used here may need to be updated depending on the location that the patch is stored on your local machine. * <strong><em>psp_data_slicing.m</em></strong> reads in the PSP MAG data and slices it into 40-second intervals. The resulting data is stored in the *_intermediates* subdirectory. * <strong><em>psp_data_labeling.m</em></strong> takes the sliced PSP MAG data from the *_intermediates* subdirectory and passes it through the MSSA algorithm and associated labeling scheme. The sub-signals are saved in <em>.mat</em> format in the *_intermediates* subdirectory. * <strong><em>psp_train_neural_net.m</em></strong> reads in the output of the sub-signal labeling from the *_intermediates* subdirectory and uses it to train the classification network described in the manuscript. * <em>'_interval...'</em>** files are used to read in specific time intervals from the PSP MAG data and generate the appropriate figures and numerical analysis. These figures are stored in the *_outputs* subdirectory.