Datasets for the development of a model for automatic trimming of transposable element sequences

Datasets for the Development of an Automatic Transposable Element Trimming Model This repository provides a collection of datasets used for the training, validation, and testing of deep learning models designed for the automatic identification and trimming of transposable elements (TEs) in genomic sequences of 15,000 base pairs. Dataset design and case composition The datasets were constructed to represent a diverse set of biologically realistic scenarios involving transposable elements embedded within genomic DNA. Each sequence belongs to one of the following cases: Case 1: DNA fragment + first TE + DNA fragment + second TE + DNA fragment Case 2: DNA fragment + first TE + DNA fragment + second TE + DNA fragment + first TE + DNA fragment Case 3: Tandemly repeated microsatellites + first TE + tandemly repeated microsatellites Case 4: DNA fragment + first TE + DNA fragment The scripts used for generate the simulated dataset are located at https://github.com/simonorozcoarias/TransposonDLToolkit/tree/main/TE_auto_trimming All TEs included in this dataset were extracted from the PanTEon Database (release 1.6.2; https://zenodo.org/records/21372179), comprising sequences derived from 1248 unique species (animals, plants and Fungi). Training dataset The following dataset was generated to train the AI models: simulated_data_28119_V3.fasta: An dataset containing 28,119 sequences, created to assess whether model performance and learned representations were reproducible at larger scales. Testing datasets Model performance was evaluated using one independent testing dataset: Dataset_testing.fasta: A testing dataset containing 2,143 sequences, with a uniform number of sequences per case. Model predictions were compared against the true TE positions to compute the R² score. manual_trimmined_seqsV2.fa:A dataset containing 50 manually-trimmed sequences from three different species: Olive (Olea europaea), Coffee rust (Hemileia vastratix), and Spotted-winged fruit fly (Drosophila suzukii). Sequence identifiers Each sequence identifier encodes detailed information about the simulated scenario and the embedded TE. Identifiers include: Case number Dfam TE identifier Initial TE position within the sequence TE length Species name All fields are separated by underscores (_).Example: >Caso1_DR003540075_s_1_0.5018_0.6526#CLASSI/LTR/GYPSY @Lupinus_angustifolius Dataset structure For each dataset, the processed data are provided as four NumPy arrays: features_data.npy:Numerical representations of the four TE-Aid plots generated for each sequence. labels_data.npy:Normalized start and end positions (values between 0 and 1) of each transposable element. case_labels.npy:Header information for each sequence (up to the # symbol). species_labels.npy:Species identifiers associated with each sequence. Trained models Four trained models are provided alongside the datasets for the following AI architectures Inpactor2_Detect: A convolutional neural network (CNN) comprising three convolutional layers with 32, 64, and 128 filters, respectively. It was originally designed to identify genomic windows containing TEs. SENMAP: A CNN architecture developed to curate LTR retrotransposon libraries. It comprises four convolutional layers with 32, 64, 128, and 256 filters, followed by three fully connected layers containing 300 neurons each. AutoTrimming: A neural network with approximately 2.3 million parameters, comprising four CNN branches followed by a fully connected neural network (FNN) with layers of 1,024, 512, 256, 64, 16, and 2 neurons, respectively. AutoTrimming2: A neural network comprising four CNN branches—one for each TE-Aid plot—based on a ResNet18-like architecture, followed by a fully connected prediction head. These trained models are compatible with the PanTEon framework (https://github.com/simonorozcoarias/PanTEon). They can therefore be downloaded and used within the software by specifying their path using the -d parameter. Reuse and applications These datasets and models are intended to support: development and benchmarking of automatic TE trimming methods, evaluation of model generalization across species, and reproducible research in transposable element annotation and deep learning–based genomic analysis.