ASKAP Science Data Processor software - ASKAPsoft Version 0.21.0

ASKAPsoft, the ASKAP Science Data Processor, provides data processing functionality, including:* Calibration* Spectral line imaging* Continuum imaging* Source detection and generation of source catalogs* Transient detectionASKAPsoft is developed as a part of the CSIRO Australian Square Kilometre Array Pathfinder (ASKAP) Science Data Processor component. ASKAPsoft is a key component in the ASKAP system. It is the primary software for storing and processing raw data, and initiating the archiving of resulting science data products into the data archive (CASDA).The processing pipelines within ASKAPsoft are largely written in C++ built on top of casacore and other third party libraries. The software is designed to be parallelised, where possible, for performance.ASKAPsoft is designed to be built and executed in a standard Unix/Linux environment and core dependencies must be fulfilled by the platform. These include, but are not limited to, a C/C++/Fortran compiler, Make, Python 2.7, Java 7 and MPI. More specific dependencies are downloaded by the ASKAPsoft build system and are installed within the ASKAPsoft development tree. Specific to the Debian platform, after a standard installation of Debian Wheezy (7.x) the following packages will need to be installed with apt-get:* g++* gfortran* openjdk-7-jdk* python-dev* flex* bison* openmpi-bin* libopenmpi-dev* libfreetype6-dev* libpng12-devMore information regarding the building, installation and running of the software can be found in the README file in the root of the file structure that forms this collection.Source code can be accessed via the links in Related Materials section.-----A large release containing a number of updates to the pipeline scriptsand to various aspects of the processing tools.Pipeline updates: * Can use AOflagger instead of cflag. * Can use continuum cubes to measure spectral indices of continuum components (using Selavy). * Bug fix, the CleanModel option of continuum-subtraction was using the wrong image name. * Allow self-calibration to use the clean model image as the model for calibration (in the manner of continuum-subtraction). * Improved continuum subtraction Selavy parameterisations, to better model continuum components. Selavy parsets are now consistent with those used for the continuum cataloguing. * Use of an alternative bandpass smoothing task - smooth_bandpass.py (instead of plot_caltable.py). * Use of an additional bandpass validation script to produce summary diagnostic plots for the bandpass solutions. * Bug fix, the bandpass table name was not set correctly when the the DO_FIND_BANDPASS switch was turned off. * Addition of the spectral measurement sets, the continuum-subtraction models/catalogues, and the spectral cube beam logs to the list of artefacts to be sent to CASDA upon pipeline completion. * Changes to some default parameters. See CHANGES file for details.Processing tasks: * MPI barrier added to the spectral imager to prevent race conditions. * Improved bandpass calibration to fix failures with SVD conversion errors. * The memory handling within linmos-mpi has been improved to reduce its footprint, making it better able to mosaic large spectral cubes. * Selavy: - now reports best component fit, regardless of the chi-squared. If poor, a new flag will be set. - If the fit fails to converge, can reduce the number of Gaussians being fit to try to get a good fit. - Bug fix, allow the curvature-map method of identifying components to better take into account the weights image associated with the image being searched. - Bug fix, Selavy, (extraction code) was fixed to allow its use on images without spectral or Stokes axes. - The SNR image produced by Selavy now has a blank string for the pixel units. - The implementation of variable threshold calculations in Selavy have been streamlined, to improve the memory use.ASKAPsoft（即ASKAP科学数据处理器）提供数据处理功能，涵盖： * 校准 * 谱线成像 * 连续谱成像 * 源探测与源目录生成 * 暂现源探测 ASKAPsoft是澳大利亚联邦科学与工业研究组织（Commonwealth Scientific and Industrial Research Organisation, CSIRO）旗下ASKAP科学数据处理器组件的组成部分，同时也是ASKAP系统的核心组件之一。它是存储与处理原始观测数据、并将生成的科学数据产品归档至数据归档系统（CASDA）的核心软件。 ASKAPsoft内的处理流水线大多基于casacore与其他第三方库，采用C++语言编写。该软件在条件允许的情况下支持并行化以提升运行性能。 ASKAPsoft设计为可在标准Unix/Linux环境中编译与运行，平台需满足核心依赖项要求，包括但不限于C/C++/Fortran编译器、Make、Python 2.7、Java 7以及MPI。更多特定依赖项将由ASKAPsoft的构建系统自动下载，并安装至ASKAPsoft的开发目录中。针对Debian平台，在完成Debian Wheezy（7.x）的标准安装后，需通过apt-get安装以下软件包： * g++ * gfortran * openjdk-7-jdk * python-dev * flex * bison * openmpi-bin * libopenmpi-dev * libfreetype6-dev * libpng12-dev 有关该软件的构建、安装与运行的更多详细信息，可参阅本数据集文件结构根目录下的README文件。源代码可通过相关资料部分的链接获取。 ----- 本次大规模发布对流水线脚本与处理工具的多个模块进行了更新。 流水线更新： * 支持使用AOflagger替代cflag进行数据标记 * 支持使用连续谱立方体测量连续谱成分的光谱指数（借助Selavy工具） * 修复了连续谱减消功能的CleanModel选项使用错误图像名称的漏洞 * 允许自校准流程使用clean模型图像作为校准模型（采用与连续谱减消一致的方式） * 优化了连续谱减消的Selavy参数配置，以更好地拟合连续谱成分，且Selavy参数文件现与连续谱编目所用参数保持一致 * 改用替代的带通平滑任务smooth_bandpass.py（原使用plot_caltable.py） * 新增额外的带通验证脚本，用于生成带通解的汇总诊断绘图 * 修复了当DO_FIND_BANDPASS开关关闭时带通表名称未正确设置的漏洞 * 新增光谱测量集、连续谱减消模型/目录以及光谱立方体波束日志至流水线运行完成后需上传至CASDA的产物列表中 * 调整了部分默认参数，具体细节请参阅CHANGES文件 处理任务更新： * 为光谱成像器添加MPI屏障，以避免竞态条件 * 优化了带通校准流程，修复了SVD转换错误导致的运行失败问题 * 优化了linmos-mpi的内存管理，降低其内存占用，使其能够更好地处理大型光谱立方体的拼接工作 * Selavy工具更新： - 无论卡方检验结果如何，均会输出最优成分拟合结果，若拟合效果不佳则会设置新的标记位 - 若拟合未收敛，可减少拟合的高斯分量数量以尝试获得有效拟合 - 修复了曲率映射法识别源时未正确考虑待搜索图像关联的权重图像的漏洞 - 修复了Selavy提取代码无法在无光谱或斯托克斯轴的图像上运行的漏洞 - Selavy生成的信噪比（SNR）图像的像素单位现设为空字符串 - 优化了Selavy中可变阈值计算的实现，以改善内存使用效率