Scalable continuous gravitational wave detection in PTA data with non-parametric red noise suppression and optimal pulsar selection

# Data Availability for: Scalable Continuous Gravitational Wave Detection in PTA Data with Non-parametric Red Noise Suppression and Optimal Pulsar Selection ## Overview This repository contains the data and analysis results associated with the paper on scalable continuous gravitational wave (CW) detection using Pulsar Timing Array (PTA) data. The dataset includes simulated NANOGrav 15-year data, Bayesian MCMC analysis chains, and processed signal data for two source sky locations with various pulsar selection methods. --- ## Directory Structure ```.├── Bayesian/ # Bayesian MCMC analysis chains├── PINT/ # Simulated NANOGrav 15-year timing data├── Signal/ # Simulated CW signals and combined data│ ├── LocationA/ # Source Location A analysis│ └── LocationB/ # Source Location B analysis├── Scripts/ # Analysis scripts└── README.md # This documentation file``` --- ## 1. Bayesian Folder This folder contains the Markov Chain Monte Carlo (MCMC) chains from Bayesian parameter estimation analysis using the `enterprise` and `PTMCMCSampler` packages. ### Subfolders | Folder Name | Description | Pulsar Selection Method (Paper Notation) ||-------------|-------------|------------------------------------------|| `pta_cw_nb_fix_DM_Sky2WW_90/` | MCMC chains with **C-SNR-90** pulsar selection | **C-SNR-90** || `pta_cw_nb_fix_DM_Sky2WW_Avg/` | MCMC chains with **C-ASNR-90** pulsar selection | **C-ASNR-90** || `pta_cw_nb_fix_DM_Sky2WW_HF60U/` | MCMC chains with **P-60** pulsar selection | **P-60** || `pta_cw_nb_fix_DM_Sky2WW_ALL/` | MCMC chains using all 68 pulsars (full PTA) | Full PTA | ### File Descriptions Each MCMC folder contains the following files: | File | Description ||------|-------------|| `chain_1.txt` | Main MCMC chain samples || `chain_*.txt` | Parallel-tempered chains at different temperatures (for Avg folder) || `pars.txt` | Parameter names corresponding to chain columns || `cov.npy` | Covariance matrix used for proposal distribution || `jumps.txt` | Jump proposal statistics || `*_jump.txt` | Individual jump proposal acceptance rates | --- ## 2. PINT Folder This folder contains simulated pulsar timing data generated using [PINT](https://github.com/nanograv/PINT) (a Python-based pulsar timing package). The data simulates NANOGrav 15-year observations. ### Subfolders | Folder Name | Description ||-------------|-------------|| `uni14_tim_fix_DM/` | Baseline timing files (.tim) for 68 pulsars with uniform 14-day cadence and fixed DM corrections || `Realizations/` | 100 independent noise realizations for each pulsar | ### File Naming Convention - **Baseline files**: `{PSR_NAME}_fake.tim` - Example: `J1909-3744_fake.tim` - **Noise realizations**: `{PSR_NAME}_fake_nrlz{N}.tim` - Example: `J1909-3744_fake_nrlz1.tim` to `J1909-3744_fake_nrlz100.tim` ### Pulsar Count - Total pulsars: **68** (matching NANOGrav 15-year narrowband dataset)- Noise realizations: **100** per pulsar --- ## 3. Signal Folder This folder contains simulated continuous gravitational wave signals and combined (signal + noise) datasets for two different source sky locations. ### Source Locations | Location | Description ||----------|-------------|| **LocationA** | First simulated CW source position || **LocationB** | Second simulated CW source position | ### Pulsar Selection Methods The data files correspond to three pulsar selection strategies described in the paper: | Data File Suffix | Paper Notation | Description ||------------------|----------------|-------------|| `sel_90` / `_90` | **C-SNR-90** | Pulsars selected by cumulative SNR threshold (90%) for each realization || `avg_snr` | **C-ASNR-90** | Pulsars selected by cumulative average SNR threshold (90%) across realizations || `high_freq_60` / `high_freq` | **P-60** | Pulsars with selection frequency ≥60 out of 100 realizations || `common` (LocationA) | Full PTA | All 68 pulsars (baseline) | ### File Structure per Location #### LocationA/ | File | Description ||------|-------------|| `SourceLocationA_Sig.mat` | MATLAB file containing the pure CW signal parameters || `selected_pulsars_90.txt` | List of pulsars selected by C-SNR-90 method || `selected_by_avg_snr.txt` | List of pulsars selected by C-ASNR-90 method || `high_frequency_psrs.txt` | List of pulsars selected by P-60 method || `Comb.hdf5` | Combined signal+noise data (full PTA) || `Comb_90.hdf5` | Combined data with C-SNR-90 selection || `Comb_Avg.hdf5` | Combined data with C-ASNR-90 selection || `Comb_hf60U.hdf5` | Combined data with P-60 selection || `resOfres_lp_r8.hdf5` | Residual-of-residuals after lowpass filtering (full PTA) || `resOfres_lp_r8_90.hdf5` | Residual-of-residuals (C-SNR-90) || `resOfres_lp_r8_Avg.hdf5` | Residual-of-residuals (C-ASNR-90) || `resOfres_lp_r8_high_freq_60.hdf5` | Residual-of-residuals (P-60) | #### LocationA/100 Realization/ Contains 100 independent realizations for statistical analysis: | File Pattern | Description ||--------------|-------------|| `resOfres_lp_r8_common_rlz{N}.hdf5` | Full PTA analysis for realization N || `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | C-ASNR-90 analysis for realization N || `resOfres_lp_r8_high_freq_60_uni_rlz{N}.hdf5` | P-60 analysis for realization N | #### LocationB/ | File | Description ||------|-------------|| `SourceLocationB_Sig.mat` | MATLAB file containing the pure CW signal parameters || `selected_psrs_90.txt` | List of pulsars selected by C-SNR-90 method || `selected_by_avg_snr.txt` | List of pulsars selected by C-ASNR-90 method || `high_frequency_psrs.txt` | List of pulsars selected by P-60 method | #### LocationB/100 Realization/ | File Pattern | Description ||--------------|-------------|| `resOfres_lp_r8_sel_90_rlz{N}.hdf5` | C-SNR-90 analysis for realization N || `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | C-ASNR-90 analysis for realization N || `resOfres_lp_r8_high_freq_rlz{N}.hdf5` | P-60 analysis for realization N | --- ## 4. Scripts Folder | File | Description ||------|-------------|| `PSR_Optimization_V.py` | Python script for pulsar optimization analysis using SNR-based selection criteria | --- ## Pulsar Selection Methods Summary | Method | Paper Notation | Description ||--------|----------------|-------------|| **Cumulative SNR 90%** | C-SNR-90 | Select pulsars contributing to 90% of the cumulative SNR² for each individual noise realization || **Cumulative Average SNR 90%** | C-ASNR-90 | Select pulsars contributing to 90% of the cumulative average SNR² computed across all realizations || **Selection Probability 60** | P-60 | Select pulsars that appear in ≥60 out of 100 realizations using the C-SNR-90 criterion | --- ## Data Format - **`.tim` files**: Standard pulsar timing format (TEMPO/TEMPO2/PINT compatible)- **`.hdf5` files**: HDF5 format containing processed residual data and metadata- **`.mat` files**: MATLAB data files containing signal parameters- **`.txt` files**: Plain text files with pulsar name lists (one per line) or MCMC chain data- **`.npy` files**: NumPy binary format for covariance matrices --- ## Software Requirements To work with this data, the following packages are recommended: - [PINT](https://github.com/nanograv/PINT) - Pulsar timing analysis- [enterprise](https://github.com/nanograv/enterprise) - PTA gravitational wave analysis- [PTMCMCSampler](https://github.com/jellis18/PTMCMCSampler) - MCMC sampling- h5py - For reading HDF5 files- scipy - For reading MATLAB files- numpy - For numerical operations# 关联论文的数据可用性说明：基于非参数红噪声抑制与最优脉冲星选择的PTA数据可扩展连续引力波探测 ## 概述 本仓库包含与基于脉冲星计时阵列（Pulsar Timing Array, PTA）数据开展可扩展连续引力波（Continuous Gravitational Wave, CW）探测的研究论文相关的数据与分析结果。本数据集涵盖模拟生成的NANOGrav 15年观测数据、贝叶斯马尔可夫链蒙特卡洛（Markov Chain Monte Carlo, MCMC）分析链，以及针对两类源天区位置、采用多种脉冲星选择方法得到的处理后信号数据。 --- ## 目录结构 .├── Bayesian/ # 贝叶斯MCMC分析链 ├── PINT/ # 模拟NANOGrav 15年计时数据 ├── Signal/ # 模拟连续引力波信号与联合数据集 │ ├── LocationA/ # 源天区A分析 │ └── LocationB/ # 源天区B分析 ├── Scripts/ # 分析脚本 └── README.md # 本说明文档 --- ## 1. 贝叶斯文件夹 本文件夹包含使用`enterprise`与`PTMCMCSampler`软件包开展贝叶斯参数估计分析所得到的马尔可夫链蒙特卡洛（MCMC）分析链。 ### 子文件夹 | 文件夹名称 | 说明 | 脉冲星选择方法（论文标注） | |-------------|-------------|------------------------------------------| | `pta_cw_nb_fix_DM_Sky2WW_90/` | 采用**C-SNR-90**脉冲星选择方法的MCMC链 | **C-SNR-90** | | `pta_cw_nb_fix_DM_Sky2WW_Avg/` | 采用**C-ASNR-90**脉冲星选择方法的MCMC链 | **C-ASNR-90** | | `pta_cw_nb_fix_DM_Sky2WW_HF60U/` | 采用**P-60**脉冲星选择方法的MCMC链 | **P-60** | | `pta_cw_nb_fix_DM_Sky2WW_ALL/` | 使用全部68颗脉冲星（全PTA阵列）的MCMC链 | 全PTA | ### 文件说明 每个MCMC文件夹均包含以下文件： | 文件 | 说明 | |------|-------------| | `chain_1.txt` | 主MCMC链采样结果 | | `chain_*.txt` | 不同温度下的并行回火链（仅Avg文件夹包含） | | `pars.txt` | 与链列对应的参数名称 | | `cov.npy` | 用于提议分布的协方差矩阵 | | `jumps.txt` | 跳跃提议统计信息 | | `*_jump.txt` | 单个跳跃提议的接受率 | --- ## 2. PINT文件夹 本文件夹包含使用[PINT](https://github.com/nanograv/PINT)（一款基于Python的脉冲星计时软件包）生成的模拟脉冲星计时数据，该数据模拟了NANOGrav的15年观测方案。 ### 子文件夹 | 文件夹名称 | 说明 | |-------------|-------------| | `uni14_tim_fix_DM/` | 包含68颗脉冲星的均匀14天观测周期与固定色散量（DM）校正的基线计时文件（.tim格式） | | `Realizations/` | 每颗脉冲星的100组独立噪声实现 | ### 文件命名规则 - **基线文件**：`{PSR_NAME}_fake.tim` 示例：`J1909-3744_fake.tim` - **噪声实现文件**：`{PSR_NAME}_fake_nrlz{N}.tim` 示例：`J1909-3744_fake_nrlz1.tim` 至 `J1909-3744_fake_nrlz100.tim` ### 脉冲星统计 - 总脉冲星数量：**68颗**（与NANOGrav 15年窄带数据集匹配） - 单颗脉冲星的噪声实现组数：**100组** --- ## 3. Signal文件夹 本文件夹包含针对两类不同源天区位置的模拟连续引力波信号，以及（信号+噪声）联合数据集。 ### 源天区位置 | 位置 | 说明 | |----------|-------------| | **LocationA** | 首个模拟连续引力波源位置 | | **LocationB** | 第二个模拟连续引力波源位置 | ### 脉冲星选择方法 本数据集文件对应论文中提及的三类脉冲星选择策略： | 数据文件后缀 | 论文标注 | 说明 | |------------------|----------------|-------------| | `sel_90` / `_90` | **C-SNR-90** | 针对每组噪声实现，选取累积信噪比平方占比达90%的脉冲星 | | `avg_snr` | **C-ASNR-90** | 针对所有噪声实现的平均累积信噪比平方，选取占比达90%的脉冲星 | | `high_freq_60` / `high_freq` | **P-60** | 选取在≥60组噪声实现中被C-SNR-90方法选中的脉冲星 | | `common`（LocationA） | 全PTA | 全部68颗脉冲星（基线方案） | ### 单一天区的文件结构 #### LocationA/ | 文件 | 说明 | |------|-------------| | `SourceLocationA_Sig.mat` | 包含纯连续引力波信号参数的MATLAB文件 | | `selected_pulsars_90.txt` | C-SNR-90方法选中的脉冲星列表 | | `selected_by_avg_snr.txt` | C-ASNR-90方法选中的脉冲星列表 | | `high_frequency_psrs.txt` | P-60方法选中的脉冲星列表 | | `Comb.hdf5` | 包含信号+噪声联合数据的HDF5文件（全PTA） | | `Comb_90.hdf5` | 采用C-SNR-90选择方法的联合数据文件 | | `Comb_Avg.hdf5` | 采用C-ASNR-90选择方法的联合数据文件 | | `Comb_hf60U.hdf5` | 采用P-60选择方法的联合数据文件 | | `resOfres_lp_r8.hdf5` | 全PTA阵列经低通滤波后的残差的残差文件 | | `resOfres_lp_r8_90.hdf5` | C-SNR-90方法下的残差的残差文件 | | `resOfres_lp_r8_Avg.hdf5` | C-ASNR-90方法下的残差的残差文件 | | `resOfres_lp_r8_high_freq_60.hdf5` | P-60方法下的残差的残差文件 | #### LocationA/100 Realization/ 包含100组独立噪声实现用于统计分析： | 文件模板 | 说明 | |--------------|-------------| | `resOfres_lp_r8_common_rlz{N}.hdf5` | 第N组噪声实现的全PTA分析结果 | | `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | 第N组噪声实现的C-ASNR-90分析结果 | | `resOfres_lp_r8_high_freq_60_uni_rlz{N}.hdf5` | 第N组噪声实现的P-60分析结果 | #### LocationB/ | 文件 | 说明 | |------|-------------| | `SourceLocationB_Sig.mat` | 包含纯连续引力波信号参数的MATLAB文件 | | `selected_psrs_90.txt` | C-SNR-90方法选中的脉冲星列表 | | `selected_by_avg_snr.txt` | C-ASNR-90方法选中的脉冲星列表 | | `high_frequency_psrs.txt` | P-60方法选中的脉冲星列表 | #### LocationB/100 Realization/ | 文件模板 | 说明 | |--------------|-------------| | `resOfres_lp_r8_sel_90_rlz{N}.hdf5` | 第N组噪声实现的C-SNR-90分析结果 | | `resOfres_lp_r8_avg_snr_rlz{N}.hdf5` | 第N组噪声实现的C-ASNR-90分析结果 | | `resOfres_lp_r8_high_freq_rlz{N}.hdf5` | 第N组噪声实现的P-60分析结果 | --- ## 4. 脚本文件夹 | 文件 | 说明 | |------|-------------| | `PSR_Optimization_V.py` | 采用基于信噪比的选择准则开展脉冲星优化分析的Python脚本 | --- ## 脉冲星选择方法汇总 | 方法 | 论文标注 | 说明 | |--------|----------------|-------------| | **累积信噪比90%法** | C-SNR-90 | 针对单组噪声实现，选取累积信噪比平方占比达90%的脉冲星 | | **累积平均信噪比90%法** | C-ASNR-90 | 针对所有噪声实现的平均累积信噪比平方，选取占比达90%的脉冲星 | | **选择频率60%法** | P-60 | 选取在≥60组噪声实现中通过C-SNR-90准则被选中的脉冲星 | --- ## 数据格式 - **`.tim`文件**：标准脉冲星计时格式（兼容TEMPO/TEMPO2/PINT） - **`.hdf5`文件**：包含处理后残差数据与元数据的HDF5格式 - **`.mat`文件**：包含信号参数的MATLAB数据文件 - **`.txt`文件**：每行一个脉冲星名称的纯文本脉冲星列表文件，或MCMC链数据文件 - **`.npy`文件**：用于存储协方差矩阵的NumPy二进制格式 --- ## 软件依赖 如需使用本数据集，推荐安装以下软件包： - [PINT](https://github.com/nanograv/PINT) - 脉冲星计时分析工具 - [enterprise](https://github.com/nanograv/enterprise) - PTA引力波分析工具 - [PTMCMCSampler](https://github.com/jellis18/PTMCMCSampler) - MCMC采样工具 - h5py - 用于读取HDF5文件的库 - scipy - 用于读取MATLAB文件的库 - numpy - 数值运算库