Scalable continuous gravitational wave detection in PTA data with non-parametric red noise suppression and optimal pulsar selection

# Data Availability for: Scalable Continuous Gravitational Wave Detection in PTA Data with Non-parametric Red Noise Suppression and Optimal Pulsar Selection ## Overview This repository contains the data and analysis results associated with the paper on scalable continuous gravitational wave (CW) detection using Pulsar Timing Array (PTA) data. The dataset includes simulated NANOGrav 15-year data, Bayesian MCMC analysis chains, and processed signal data for two source sky locations with various pulsar selection methods. --- ## Directory Structure ```.├── Bayesian/ # Bayesian MCMC analysis chains├── PINT/ # Simulated NANOGrav 15-year timing data├── Signal/ # Simulated CW signals and combined data│ ├── LocationA/ # Source Location A analysis│ └── LocationB/ # Source Location B analysis├── Scripts/ # Analysis scripts└── README.md # This documentation file``` --- ## 1. Bayesian Folder This folder contains the Markov Chain Monte Carlo (MCMC) chains from Bayesian parameter estimation analysis using the `enterprise` and `PTMCMCSampler` packages. ### Subfolders | Folder Name | Description | Pulsar Selection Method (Paper Notation) ||-------------|-------------|------------------------------------------|| `pta_cw_nb_fix_DM_Sky2WW_90/` | MCMC chains with **C-SNR-90** pulsar selection | **C-SNR-90** || `pta_cw_nb_fix_DM_Sky2WW_Avg/` | MCMC chains with **C-ASNR-90** pulsar selection | **C-ASNR-90** || `pta_cw_nb_fix_DM_Sky2WW_HF60U/` | MCMC chains with **P-60** pulsar selection | **P-60** || `pta_cw_nb_fix_DM_Sky2WW_ALL/` | MCMC chains using all 68 pulsars (full PTA) | Full PTA | ### File Descriptions Each MCMC folder contains the following files: | File | Description ||------|-------------|| `chain_1.txt` | Main MCMC chain samples || `chain_*.txt` | Parallel-tempered chains at different temperatures (for Avg folder) || `pars.txt` | Parameter names corresponding to chain columns || `cov.npy` | Covariance matrix used for proposal distribution || `jumps.txt` | Jump proposal statistics || `*_jump.txt` | Individual jump proposal acceptance rates | --- ## 2. PINT Folder This folder contains simulated pulsar timing data generated using [PINT](https://github.com/nanograv/PINT) (a Python-based pulsar timing package). The data simulates NANOGrav 15-year observations. ### Subfolders | Folder Name | Description ||-------------|-------------|| `uni14_tim_fix_DM/` | Baseline timing files (.tim) for 68 pulsars with uniform 14-day cadence and fixed DM corrections || `Realizations/` | 100 independent noise realizations for each pulsar | ### File Naming Convention - **Baseline files**: `{PSR_NAME}_fake.tim` - Example: `J1909-3744_fake.tim` - **Noise realizations**: `{PSR_NAME}_fake_nrlz{N}.tim` - Example: `J1909-3744_fake_nrlz1.tim` to `J1909-3744_fake_nrlz100.tim` ### Pulsar Count - Total pulsars: **68** (matching NANOGrav 15-year narrowband dataset)- Noise realizations: **100** per pulsar --- ## 3. Signal Folder This folder contains simulated continuous gravitational wave signals and combined (signal + noise) datasets for two different source sky locations. ### Source Locations | Location | Description ||----------|-------------|| **LocationA** | First simulated CW source position || **LocationB** | Second simulated CW source position | ### Pulsar Selection Methods The data files correspond to three pulsar selection strategies described in the paper: | Data File Suffix | Paper Notation | Description ||------------------|----------------|-------------|| `sel_90` / `_90` | **C-SNR-90** | Pulsars selected by cumulative SNR threshold (90%) for each realization || `avg_snr` | **C-ASNR-90** | Pulsars selected by cumulative average SNR threshold (90%) across realizations || `high_freq_60` / `high_freq` | **P-60** | Pulsars with selection frequency ≥60 out of 100 realizations || `common` (LocationA) | Full PTA | All 68 pulsars (baseline) | ### File Structure per Location #### LocationA/ | File | Description ||------|-------------|| `SourceLocationA_Sig.mat` | MATLAB file containing the pure CW signal parameters || `selected_pulsars_90.txt` | List of pulsars selected by C-SNR-90 method || `selected_by_avg_snr.txt` | List of pulsars selected by C-ASNR-90 method || `high_frequency_psrs.txt` | List of pulsars selected by P-60 method || `Comb.hdf5` | Combined signal+noise data (full PTA) || `Comb_90.hdf5` | Combined data with C-SNR-90 selection || `Comb_Avg.hdf5` | Combined data with C-ASNR-90 selection || `Comb_hf60U.hdf5` | Combined data with P-60 selection || `resOfres_lp_r8.hdf5` | Residual-of-residuals after lowpass filtering (full PTA) || `resOfres_lp_r8_90.hdf5` | Residual-of-residuals (C-SNR-90) || `resOfres_lp_r8_Avg.hdf5` | Residual-of-residuals (C-ASNR-90) || `resOfres_lp_r8_high_freq_60.hdf5` | Residual-of-residuals (P-60) | #### LocationA/100 Realization/ Contains 100 independent realizations for statistical analysis: | File Pattern | Description ||--------------|-------------|| `resOfres_lp_r8_common_rlz{N}.hdf5` | Full PTA analysis for realization N || `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | C-ASNR-90 analysis for realization N || `resOfres_lp_r8_high_freq_60_uni_rlz{N}.hdf5` | P-60 analysis for realization N | #### LocationB/ | File | Description ||------|-------------|| `SourceLocationB_Sig.mat` | MATLAB file containing the pure CW signal parameters || `selected_psrs_90.txt` | List of pulsars selected by C-SNR-90 method || `selected_by_avg_snr.txt` | List of pulsars selected by C-ASNR-90 method || `high_frequency_psrs.txt` | List of pulsars selected by P-60 method | #### LocationB/100 Realization/ | File Pattern | Description ||--------------|-------------|| `resOfres_lp_r8_sel_90_rlz{N}.hdf5` | C-SNR-90 analysis for realization N || `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | C-ASNR-90 analysis for realization N || `resOfres_lp_r8_high_freq_rlz{N}.hdf5` | P-60 analysis for realization N | --- ## 4. Scripts Folder | File | Description ||------|-------------|| `PSR_Optimization_V.py` | Python script for pulsar optimization analysis using SNR-based selection criteria | --- ## Pulsar Selection Methods Summary | Method | Paper Notation | Description ||--------|----------------|-------------|| **Cumulative SNR 90%** | C-SNR-90 | Select pulsars contributing to 90% of the cumulative SNR² for each individual noise realization || **Cumulative Average SNR 90%** | C-ASNR-90 | Select pulsars contributing to 90% of the cumulative average SNR² computed across all realizations || **Selection Probability 60** | P-60 | Select pulsars that appear in ≥60 out of 100 realizations using the C-SNR-90 criterion | --- ## Data Format - **`.tim` files**: Standard pulsar timing format (TEMPO/TEMPO2/PINT compatible)- **`.hdf5` files**: HDF5 format containing processed residual data and metadata- **`.mat` files**: MATLAB data files containing signal parameters- **`.txt` files**: Plain text files with pulsar name lists (one per line) or MCMC chain data- **`.npy` files**: NumPy binary format for covariance matrices --- ## Software Requirements To work with this data, the following packages are recommended: - [PINT](https://github.com/nanograv/PINT) - Pulsar timing analysis- [enterprise](https://github.com/nanograv/enterprise) - PTA gravitational wave analysis- [PTMCMCSampler](https://github.com/jellis18/PTMCMCSampler) - MCMC sampling- h5py - For reading HDF5 files- scipy - For reading MATLAB files- numpy - For numerical operations