Simulated NGS read datasets for prediction of novel fungal pathogens and multiple pathogen classes

This repository contains simulated Illumina read datasets for novel fungal pathogen prediction and real-time detection of multiple pathogen classes. They were used to train the models hosted at https://zenodo.org/record/5711877.<br> The reads were simulated with Mason (https://www.seqan.de/apps/mason/) from genomes downloaded from NCBI, based on metadata stored in a manually curated database (https://zenodo.org/record/5711852). We provide the following: 1) An rds file describing assignment of fungal species from the database (https://zenodo.org/record/5711852) to training, validation and test sets (TrainValTest_fungi.rds). 2) Fungal validation and test sets. Each contains 1.25 million, 250bp-long reads simulated from non-overlapping sets of human ("pathogenic") or non-human ("nonpathogenic") pathogens. The test set contains paired reads ("_1" and "_2" for the first and second mate). The number of reads per species is proportional to the respective genome length. 3) Fungal training sets. They contain 250bp-long reads simulated from species not present in the validation or test sets. There are four variants:<br> 3a) "low-coverage, linear" - 20 million reads, number of reads per species proportional to genome length<br> 3b) "low-coverage, logarithmic" - 20 million reads, number of reads per species proportional to the logarithm of genome length ("log")<br> 3c) "high-coverage, linear" - 240 million reads, number of reads per species proportional to genome length ("24")<br> 3d) "high-coverage, logarithmic" - 240 million reads, number of reads per species proportional to the logarithm of genome length ("24log") 4) Training, validation and test sets for the multiclass models. They should be used together with the "pathogenic" read sets hosted at https://zenodo.org/record/4456857. Here, we share sets for two of the four total classes:<br> 4a) The 'non-pathogen' class is a mixture of "nonpathogenic" biacterial and viral read sets, concatenated and downsampled to the original read number (20M for training, 1.25M for validation and test). The training and validation sets contain mixed-length (25-20bp) simulated subreads (original sets hosted here: https://zenodo.org/record/4456857). The test set contains 250bp long reads based on the test sets from here: https://zenodo.org/record/3678563 and here: https://zenodo.org/record/4312525; it was also sorted by species.<br> 4b) Mixed-length versions of the "pathogenic" fungal training and validation sets, prepared by random shortening of the "low-coverage" read sets in the "linear" (_rn_) and "logarithmic" (_rn_*log_) flavours. See also the preprint: https://www.biorxiv.org/content/10.1101/2021.11.30.470625v1本仓库包含用于新型真菌病原体预测及多类病原体实时检测的模拟Illumina测序读段数据集，相关数据集曾用于训练托管于https://zenodo.org/record/5711877的模型。 这些读段由Mason（https://www.seqan.de/apps/mason/）基于从NCBI下载的基因组模拟生成，所用基因组的元数据存储于经人工整理审核的数据库（https://zenodo.org/record/5711852）。 本仓库提供以下内容： 1. 一个rds格式文件，用于描述该数据库（https://zenodo.org/record/5711852）中的真菌物种如何划分为训练集、验证集与测试集（TrainValTest_fungi.rds）。 2. 真菌验证集与测试集：二者均包含125万条长度为250bp的读段，均从互不重叠的人类（"致病性"）或非人类（"非致病性"）病原体的基因组中模拟得到。测试集包含配对读段（正向读段后缀为"_1"，反向读段后缀为"_2"）。每个物种的读段数量与对应基因组的长度成正比。 3. 真菌训练集：包含从验证集与测试集之外的物种基因组中模拟得到的250bp读段，共包含四种变体： 3a) 低覆盖度线性分布：共计2000万条读段，各物种的读段数量与基因组长度成正比 3b) 低覆盖度对数分布：共计2000万条读段，各物种的读段数量与基因组长度的对数值成正比（简称log分布） 3c) 高覆盖度线性分布：共计2.4亿条读段，各物种的读段数量与基因组长度成正比（简称24分布） 3d) 高覆盖度对数分布：共计2.4亿条读段，各物种的读段数量与基因组长度的对数值成正比（简称24log分布） 4. 多分类模型所用的训练集、验证集与测试集：需与托管于https://zenodo.org/record/4456857的"致病性"读段数据集配合使用。本次公开的数据集涵盖全部四类中的两类： 4a) "非病原体"类：由"非致病性"细菌与病毒读段数据集混合后拼接，并下采样至原始读段数量（训练集2000万条，验证集与测试集各125万条）。其训练集与验证集包含混合长度（25-20bp）的模拟子读段，原始数据集托管于https://zenodo.org/record/4456857。其测试集的读段长度为250bp，基于https://zenodo.org/record/3678563和https://zenodo.org/record/4312525中的测试集构建，并按物种进行了排序。 4b) "致病性"真菌训练集与验证集的混合长度版本：通过对"低覆盖度"读段集进行随机截短得到，分别对应"线性分布"（后缀为_rn）与"对数分布"（后缀为_rn*log）两种类型。 详见预印本：https://www.biorxiv.org/content/10.1101/2021.11.30.470625v1