Datasets for the development of a model for automatic trimming of transposable element sequences

Datasets for the Development of an Automatic Transposable Element Trimming Model This repository provides a collection of datasets used for the training, validation, and testing of a deep learning model designed for the automatic identification and trimming of transposable elements (TEs) in genomic sequences of 15,000 base pairs. Dataset design and case composition The datasets were constructed to represent a diverse set of biologically realistic scenarios involving transposable elements embedded within genomic DNA. Each sequence belongs to one of the following cases: Case 1: DNA fragment + first TE + DNA fragment + second TE + DNA fragment Case 2: DNA fragment + first TE + DNA fragment + second TE + DNA fragment + first TE + DNA fragment Case 3: Tandemly repeated microsatellites + first TE + tandemly repeated microsatellites Case 4: DNA fragment + first TE + DNA fragment All transposable elements included in these datasets were extracted from a curated TE library (r.1.5_all.fasta), comprising sequences derived from 903 unique species. Training datasets Two training datasets were generated to evaluate model robustness and reproducibility with increasing data volume: dataset_9218.fasta:The initial training dataset, containing 9,218 sequences. dataset_27165.fasta:An expanded dataset containing 27,165 sequences, created to assess whether model performance and learned representations were reproducible at larger scales. Testing datasets Model performance was evaluated using independent testing datasets: dataset_testing.fasta:A testing dataset containing 2,143 sequences, with a uniform number of sequences per case. Model predictions were compared against the true TE positions to compute the R² score. Species-independent evaluation dataset:A subdataset derived from dataset_27165.fasta, containing only sequences from species not present in the 9,218-sequence training dataset. This dataset was used to evaluate the model’s ability to generalize to previously unseen species. Sequence identifiers Each sequence identifier encodes detailed information about the simulated scenario and the embedded TE. Identifiers include: Case number Dfam TE identifier Initial TE position within the sequence TE length Species name All fields are separated by underscores (_).Example: >Caso1_DR003540075_s_1#CLASSI/LTR/GYPSY_6028_3762_Lupinus_angustifolius Dataset structure For each dataset, the processed data are provided as four NumPy arrays: features_data.npy:Numerical representations of the four TE-Aid plots generated for each sequence. labels_data.npy:Normalized start and end positions (values between 0 and 1) of each transposable element. case_labels.npy:Header information for each sequence (up to the # symbol). species_labels.npy:Species identifiers associated with each sequence. Trained models Two trained models are provided alongside the datasets. Each model consists of: Four convolutional neural network (CNN) branches, one per TE-Aid plot, with a ResNet18-like architecture. A fully connected neural network (FNN) for final prediction. The available models are: model_trained_27165: Trained using the 27,165-sequence dataset. model_trained_9218: Trained using the 9,218-sequence dataset. Reuse and applications These datasets and models are intended to support: development and benchmarking of automatic TE trimming methods, evaluation of model generalization across species, and reproducible research in transposable element annotation and deep learning–based genomic analysis.