Simulated NGS read datasets for prediction of novel fungal pathogens and multiple pathogen classes

This repository contains simulated Illumina read datasets for novel fungal pathogen prediction and real-time detection of multiple pathogen classes. They were used to train the models hosted at https://zenodo.org/record/5711877.
The reads were simulated with Mason (https://www.seqan.de/apps/mason/) from genomes downloaded from NCBI, based on metadata stored in a manually curated database (https://zenodo.org/record/5846345).

We provide the following:

1) An rds file describing assignment of fungal species from the database (https://zenodo.org/record/5846345) to training, validation and test sets (TrainValTest_fungi.rds). A second rds file (TrainValTest_temporal.rds) includes species added within 12 weeks after the original datasets were compiled. Those species were used for a temporal benchmark.

2) Fungal validation and test sets. Each contains 1.25 million, 250bp-long reads simulated from non-overlapping sets of human ("pathogenic") or non-human ("nonpathogenic") pathogens. The test set contains paired reads ("_1" and "_2" for the first and second mate). The number of reads per species is proportional to the respective genome length. An additional, temporal test set (*temporal*fasta.gz) includes 15 species added after 12 weeks from the consturction of the original datasets.

3) Fungal training sets. They contain 250bp-long reads simulated from species not present in the validation or test sets. There are four variants:
3a) "low-coverage, linear" - 20 million reads, number of reads per species proportional to genome length
3b) "low-coverage, logarithmic" - 20 million reads, number of reads per species proportional to the logarithm of genome length ("log")
3c) "high-coverage, linear" - 240 million reads, number of reads per species proportional to genome length  ("24")
3d) "high-coverage, logarithmic" - 240 million reads, number of reads per species proportional to the logarithm of genome length  ("24log")

4) Training, validation and test sets for the multiclass models. They should be used together with the "pathogenic" read sets hosted at https://zenodo.org/record/4456857. Here, we share sets for two of the four total classes:
4a) The 'non-pathogen' class is a mixture of "nonpathogenic" biacterial and viral read sets, concatenated and downsampled to the original read number (20M for training, 1.25M for validation and test). The training and validation sets contain mixed-length (25-20bp) simulated subreads (original sets hosted here: https://zenodo.org/record/4456857). The test set contains 250bp long reads based on the test sets from here: https://zenodo.org/record/3678563 and here: https://zenodo.org/record/4312525; it was also sorted by species.
4b) Mixed-length versions of the "pathogenic" fungal training and validation sets, prepared by random shortening of the "low-coverage" read sets in the "linear" (_rn_) and "logarithmic" (_rn_*log_) flavours.

See also the preprint: https://www.biorxiv.org/content/10.1101/2021.11.30.470625