Dynamic Spectra associated with the paper "Transient Blurring of the Scintillation Arc of Pulsar B1737+13"

contents of the 0README file (as of 2025-02-26) This is a very basic documentation file for the dynamic spectrum repositorythat we have created to accompany the publication of our paper "Transient Blurring of the Scintillation Arc of Pulsar B1737+13"authors: Yen-Hua Chen, Sammy Siegel, Daniel Baker, Ue-Li Pen, and Dan Stinebring We plan to submit this to the Astrophysical Journal. This Zenodo DOI will bereferenced in that paper when it is published. This document was written by Dan Stinebring on Feb. 4, 2025 and finalized on Feb. 26, 2025. An arXiv version of our paper can be found here: https://arxiv.org/abs/2412.10323 First, these dynamic spectra, are the data behind the "Transient Blurring ..."paper and ALSO the data that were analyzed in this earlier paper: Hemberger and Stinebring 2008 "Time Variability of Interstellar Scattering and Improvements to Pulsar Timing"https://ui.adsabs.harvard.edu/abs/2008ApJ...674L..37H/abstract Second, we have done a decent job preparing this data set, but we are not claiming thatthis work is complete or free of problems. In particular, there is un-mitigated radio frequency interference (RFI) in the data still, although the "Cleaned" version ofthe data set is definitely an improvement on the "Raw" version in this regard. Onthe other hand, the Cleaned version was done by setting affected rows and columns ofthe dynamic spectrum (DS) array to not-a-number (NaN) values. More sophisticated RFI excision/treatment options could be applied to the Raw data files. The Cleaned files differ from the Raw files in two other aspects. First, the FITS headershave been fixed up (astropy.io.fits.verify)https://docs.astropy.org/en/stable/io/fits/usage/verification.htmlso that they are strictly conformant with the FITS standard. Second, the subintegration-to-subintegration modulation present in the Raw data setsdue to the relatively small number of pulse periods included in each10s subint for this P = 0.803 s period pulsar has been removed. This wasdone by dividing by a singular value decomposition (mode 0) model of the DSarray. An example of the procedure followed in doing this is given inthe Jupyter notebook: SVDExample.ipynb in the top-level directory.You will need to rename the file that is accessed in that notebook since thefile naming convention has now changed. Regarding that file name change: internally (before Jan. 20, 2025) we were usinga naming convention like this: B1737+13.wapp1.53830.0.dyn.fitsB1737+13.wapp2.53830.0.dyn.fitsB1737+13.wapp3.53830.0.dyn.fitsB1737+13.wapp4.53830.0.dyn.fits... We changed this to be clearer and easier to search ...B1737+13_53830.36_1175MHz_AO.fitsB1737+13_53830.36_1360MHz_AO.fitsB1737+13_53830.36_1400MHz_AO.fitsB1737+13_53830.36_1440MHz_AO.fits... and that is the pattern that we have used for filenames in both the Cleaned and Raw directories, with a _raw.fits appended to each file in the Raw directory. DIAGNOSTIC PLOTS----------------These are included to give you a quick look at the 144 data files in the dataset. However,the program used to produce the plots is legacy IDL software from Dan Stinebring's lab(~ 1998 - 2005) and does not have many of the state-of-art enhancements provided inDaniel Reardon's scintools software suite: https://github.com/danielreardon/scintools So, look at these plots if they are useful to you, but don't consider themresearch-ready for any further purposes. NOTE again: the Cleaned files have NaNs marking excised channels and subints. You will needto deal with those in some fashion when proceeding with your own analysis. The Rawfiles do not have any NaNs in them, but they have significant unexcised RFI atsome epochs. The Diagnostic Plots show all four 50-MHz bands on one page. There are 36 pages per PDFbecause there are 36 epochs of observation with nearly weekly spacing. The header at thetop of each plot sub-panel lists the pulsar (B1737+13), the MJD, and the year-frac of thatepoch. The IDL command used to created the combined PDF for both the raw & cleaned files is similar to thisIDL> dyn_ss_plot,files_clean,/nohanning,/fullsec,outfile='0Output/Diagnostic',nrows=2,ncols=2,/compact The raw files have numerous non-standard aspects to their FITS headers. As noted above, the cleaned files havehad these problems fixed by the astropy verify() method https://docs.astropy.org/en/stable/io/fits/usage/verification.html (accessed 2025-02-26) e.g. (from other astropy fits documentation) from astropy.io import fitsfilename = fits.util.get_testdata_filepath('verify.fits') Non-standard FITS keywords--------------------------T_INT = 10.0 subintegration time NBINS = 128 number of pulse phase bins (used in producing dynamic spectra) NCHANS = 2048 number of frequency channels ORIGIN = 'Arecibo ' telescope of origin (this is actually a standard FITS keyword) SCINTCUB= 'Scintcube' Program used to create dynamic spectrum (IDL code; Stinebring lab) SPECTROM= 'WAPP ' Spectrometer used at Arecibo TTOT = 3600.00000000000000000000 total integration time (s)截至2025年2月26日的0README文件内容 本文件为我们为配合论文《脉冲星B1737+13闪烁弧的暂态模糊》（"Transient Blurring of the Scintillation Arc of Pulsar B1737+13"）发表而创建的动态光谱库基础说明文档，作者：Yen-Hua Chen、Sammy Siegel、Daniel Baker、Ue-Li Pen、Dan Stinebring。 我们计划将该论文投稿至《天体物理学杂志》（Astrophysical Journal），论文正式发表后，将在文中引用本数据集的Zenodo DOI标识。 本说明文档由Dan Stinebring于2025年2月4日撰写，2025年2月26日定稿。 本论文的arXiv预印本可通过以下链接获取：https://arxiv.org/abs/2412.10323 其一，本数据集包含的动态光谱数据，既是上述《脉冲星B1737+13闪烁弧的暂态模糊》论文的支撑数据，同时也是早期论文《星际散射的时间变化特性与脉冲星计时优化》（Hemberger & Stinebring, 2008）的分析数据集，该论文链接：https://ui.adsabs.harvard.edu/abs/2008ApJ...674L..37H/abstract。 其二，本数据集的预处理工作已完成基础搭建，但我们并未宣称其已完全完善或不存在瑕疵。具体而言，数据中仍存在未被抑制的射频干扰（Radio Frequency Interference, RFI），尽管“已清洗（Cleaned）”版本的数据在该方面相较于“原始（Raw）”版本已有显著改善。需说明的是，已清洗版本通过将动态光谱（Dynamic Spectrum, DS）数组中受干扰的行与列赋值为非数值（Not-a-Number, NaN）来标记受影响区域，而原始数据文件仍可采用更先进的射频干扰切除或处理方案。 已清洗版本与原始版本另有两处差异。其一，已修复FITS文件头（使用astropy.io.fits.verify工具，参考文档：https://docs.astropy.org/en/stable/io/fits/usage/verification.html），使其严格符合FITS标准规范。 其二，针对周期P=0.803秒的该脉冲星，原始数据集中每个10秒子积分段包含的脉冲周期数较少，导致子积分间存在幅度调制，该调制已被移除。具体方法为将动态光谱数组除以基于奇异值分解（Singular Value Decomposition, SVD，模式0）构建的模型。相关处理流程的示例可通过顶层目录下的Jupyter笔记本SVDExample.ipynb查看，由于当前文件名规则已更新，你需要修改该笔记本中调用的文件名。 关于文件名规则变更：在2025年1月20日之前，我们内部使用如下命名格式： B1737+13.wapp1.53830.0.dyn.fits B1737+13.wapp2.53830.0.dyn.fits B1737+13.wapp3.53830.0.dyn.fits B1737+13.wapp4.53830.0.dyn.fits…… 为提升可读性与检索便利性，我们将其调整为如下格式： B1737+13_53830.36_1175MHz_AO.fits B1737+13_53830.36_1360MHz_AO.fits B1737+13_53830.36_1400MHz_AO.fits B1737+13_53830.36_1440MHz_AO.fits…… 当前已清洗与原始数据集目录下的文件均采用该命名规则，其中原始目录下的文件名会追加`_raw.fits`后缀。 --- 诊断绘图 这些绘图用于快速预览数据集中的144个数据文件。但用于生成绘图的程序为Dan Stinebring实验室的遗留IDL软件（开发于1998-2005年左右），并未集成Daniel Reardon的scintools软件套件中的诸多前沿增强功能，相关套件链接：https://github.com/danielreardon/scintools。若绘图对你有所帮助可参考使用，但请勿将其作为可直接用于科研的成品。 再次提醒：已清洗版本的文件中使用NaN值标记了已切除的通道与子积分段，你在后续自主分析时需对该标记进行相应处理。原始文件中不存在NaN值，但在部分观测时段仍存在未被切除的显著射频干扰。 诊断绘图每页展示全部4个50MHz频段的数据，每份PDF包含36页，对应36个几乎按周间隔的观测时段。每个绘图子面板顶部的表头会标注脉冲星名称（B1737+13）、修正儒略日（MJD）以及该观测时段的年份分数。 用于生成原始与已清洗数据集合并PDF的IDL命令示例如下： `IDL> dyn_ss_plot,files_clean,/nohanning,/fullsec,outfile='0Output/Diagnostic',nrows=2,ncols=2,/compact` 原始文件的FITS文件头存在多处非标准问题，如前所述，已清洗版本的文件头已通过astropy的verify()方法完成修复（参考文档：https://docs.astropy.org/en/stable/io/fits/usage/verification.html，访问时间：2025年2月26日）。示例代码（来自astropy官方FITS文档）如下： python from astropy.io import fits filename = fits.util.get_testdata_filepath('verify.fits') --- 非标准FITS关键字 T_INT = 10.0 子积分时长 NBINS = 128 脉冲相位分箱数（用于生成动态光谱） NCHANS = 2048 频率通道数 ORIGIN = 'Arecibo ' 观测望远镜（该关键字实为标准FITS关键字） SCINTCUB= 'Scintcube' 用于生成动态光谱的程序（IDL代码，Stinebring实验室开发） SPECTROM= 'WAPP ' 阿雷西博天文台使用的光谱仪 TTOT = 3600.00000000000000000000 总积分时长（单位：秒）