UltraSynBERT_All_Archives

Decoding the RNA interactome by UltraSynBERT The official repository for the article "Decoding the RNA interactome by UltraSynBERT". Overview RNA encodes complex molecular behaviors that underlie binding, regulation, and cellular fate. Current RNA language models have been shaped primarily by endogenous transcripts, leaving the potential of exogenous RNAs to extend model boundaries underexplored. We present UltraSynBERT, an RNA language model pre-trained on synthetic RNA binding landscapes generated from in vitro selection experiments. By learning sequence features associated with RNA-target interactions, the model enables prediction of RNA sequence-dependent properties across in vitro binding datasets and in vivo regulatory contexts. Fine-tuned on in vitro SELEX datasets, the model identifies RNA aptamers for diverse targets, including small molecules, proteins, cells, and tissues. The model also predicts tissue specificity for millions of RNA species across 22 human organs based on their 3’-UTR sequences, captures human-pathogenic viral RNA tropism, recognizes RNA base modifications, and characterizes SARS-CoV-2 replicase binding with single-base resolution features. UltraSynBERT Archives 1. Pre-training datasets and models In this repository, you will find the following pre-training datasets and pre-trained model checkpoints: Model Name Parameters Pre-training Data RNA Species Number Description Checkpoint UltraSynBERT 33.5M UltraSelex SiR-B 10 million pre-training from scratch UltraSynBERT.pkl UltraSynBERTbase-only 33.5M UltraSelex SiR-B 10 million pre-training from scratch UltraSynBERT_base.pkl UltraSynBERTmotif-only 33.5M UltraSelex SiR-B 10 million pre-training from scratch UltraSynBERT_motif.pkl UltraSynBERTsource_SiR 33.5M SELEX SiR 10 million pre-training from scratch UltraSynBERT_source_SiR.pkl UltraSynBERTsource-nsp12 33.5M SELEX nsp12 10 million pre-training from scratch UltraSynBERT_source_nsp12.pkl UltraSynBERTmolecules 33.5M 12 SELEX targets’ training sets 3.24 million continued pre-training the basic UltraSynBERT model UltraSynBERT_molecule.pkl UltraSynBERT3UTR 33.5M The preliminary 3’-UTR full training datasets 1.69 million continued pre-training the basic UltraSynBERT model UltraSynBERT_3UTR.pkl UltraSynBERTplus 33.5M 12 SELEX targets’ training sets and the preliminary 3’-UTR full training datasets 4.93 million continued pre-training the basic UltraSynBERT model UltraSynBERT_plus.pkl UltraSynBERTRIP 33.5M Human RIP-Seq in ENCODE 6.88 million continued pre-training the basic UltraSynBERT model UltraSynBERT_RIP.pkl NOTE: The pretraining data is located in the Dataset Availability\1. Dataset for pre-training folder. The pretrained models can be found in the Model Availability\1. Pre-training models and Model Availability\2. Continued pre-training models folders. 2. Downstream task datasets and fine-tuned models Additionally, this official repository also provides all datasets for downstream tasks and supervised fine-tuned models based on the base UltraSynBERT for downstream applications, including: RNA aptamer identification models targeting small molecules, proteins, cells, and tissues (12 specialized datasets), and tissue-specificity prediction for human mRNA 3'-UTR regions. Task Type Data Type Dataset Number Dataset SiR-binding prediction In vitro 3 (1) UltraSelex-SiR-B.csv (2) SELEX-SiR-whole.csv (3) SELEX-SiR-exclusive.csv In vitro RNA-target binding prediction In vitro 12 (1) SELEX-small-molecule-DAse.csv; (2) SELEX-small-molecule-BC.csv: (3) SELEX-small-molecule-PR.csv; (4) SELEX-small-molecule-MI.csv: (5) SELEX-protein-TARDBP.csv: (6) SELEX-protein-RT.csv; (7) SELEX-protein-RBM24.csv; (8) SELEX-protein-S15.csv; (9) SELEX-(multi)cellular-ISLETS.csv; (10) SELEX-(multi)cellular-MDSC.csv; (11) SELEX-(multi)cellular-CHO-K1.csv;(12) SELEX-(multi)cellular-TNBC.csv In vivo RNA-protein interactions prediction (human source) In vivo 11 (1) 16_ICLIP_hnRNPC_Hela_iCLIP_all_clusters_sequences.csv; (2) 17_ICLIP_HNRNPC_hg19_sequences.csv;(3) 18_ICLIP_hnRNPL_Hela_group_3975_all-hnRNPL-Hela-hg19_sum_G_hg19--ensembl59_from_2337-2339-741_bedGraph-cDNA-hits-in-genome_sequences.csv; (4) 19_ICLIP_hnRNPL_U266_group_3986_all-hnRNPL-U266-hg19_sum_G_hg19--ensembl59_from_2485_bedGraph-cDNA-hits-in-genome_sequences.csv; (5) 20_ICLIP_hnRNPlike_U266_group_4000_all-hnRNPLlike-U266-hg19_sum_G_hg19--ensembl59_from_2342-2486_bedGraph-cDNA-hits-in-genome_sequences.csv; (6) 22_ICLIP_NSUN2_293_group_4007_all-NSUN2-293-hg19_sum_G_hg19--ensembl59_from_3137-3202_bedGraph-cDNA-hits-in-genome_sequences.csv; (7) 27_ICLIP_TDP43_hg19_sequences.csv; (8) 28_ICLIP_TIA1_hg19_sequences.csv; (9) 29_ICLIP_TIAL1_hg19_sequences.csv; (10) 30_ICLIP_U2AF65_Hela_iCLIP_ctrl_all_clusters_sequences.csv; (11) 31_ICLIP_U2AF65_Hela_iCLIP_ctrl+kd_all_clusters_sequences.csv In vivo RNA-protein interactions prediction (mouse source) In vivo 11 (1) CLIP-mouse-EZH2-sequences.csv; (2) CLIP-mouse-FUS-sequences.csv; (3) CLIP-mouse-HNRNPR-sequences.csv; (4) CLIP-mouse-LIN28A-sequences.csv; (5) CLIP-mouse-RBFOX2-sequences.csv; (6) CLIP-mouse-RBM10-sequences.csv; (7) CLIP-mouse-SRSF2-sequences.csv; (8) CLIP-mouse-SRSF3-sequences.csv; (9) CLIP-mouse-TARDBP-sequences.csv; (10) CLIP-mouse-U2AF2-sequences.csv; (11) CLIP-mouse-YTHDC2-sequences.csv modification In vivo 13 m6A: (1) m6A-A549.csv; (2) m6A-CD8T.csv; (3) m6A-ESC.csv; (4) m6A-HCT116.csv; (5) m6A-HEK293.csv; (6) m6A-HEK293T.csv; (7)m6A-Hela.csv; (8) m6A-HepG2.csv; (9) m6A-MOLM13.csv m1A: m1A_dataset_with_split.csv m5C: m5c_dataset_with_split.csv m6Am: m6Am_dataset_with_split.csv pseudouridine: pseudouridine_dataset_with_split.csv 3'UTR tissue-specific recognition In vivo 1 human_22_tissue_three_terminal_UTR.csv (Transcriptome-wide split) human_22_tissue_three_terminal_UTR_gene_split.csv (Gene-disjoint split) Zero-shot RNA mutation effect prediction targeting SARs-CoV-2 replicase nsp12 In vitro The wild-type sequence and seven mutated sequences RNA-mutation.fasta RNA post-transcriptional regulation–related tasks in vitro/In vivo 4 (1) miRNA _interactions.csv; (2) mRNA_subcellular_localization.csv; (3) Polyadenylation_signal.csv; (4) RNA_stability.csv human-pathogenic RNA viruses In vivo 1 virus.csv RNA switch-mediated regulation In vitro 1 ProgrammableRNASwitches.csv NOTE: The datasets for downstream tasks are in the Dataset Availability\2. Dataset for fine-tuning folder. The fine-tuned models for downstream tasks are available in the Model Availability\3. Fine-tuned models folder. The following models were obtained by fine-tuning the base UltraSynBERT checkpoint on downstream tasks, including in vitro RNA-target interaction prediction and in vivo human 3′-UTR tissue-specific recognition. Task Type Target Type Name Dataset Method / Split Length (nt) PMID / Source Fine-tuned Model Checkpoint RNA-target interaction Small molecules Benzopyrylium-coumarin fluorophores BC SELEX 98–106 34309994 UltraSynBERT_System_Ranking_for_BC.pkl RNA-target interaction Small molecules Paromomycin PR Capture-SELEX 119–125 30957848 UltraSynBERT_System_Ranking_for_PR.pkl RNA-target interaction Small molecules Maleimide involved in Diels-Alderase DAse SELEX 152–162 24157838 UltraSynBERT_System_Ranking_for_DAse.pkl RNA-target interaction Small molecules Mechanistic inhibitor of serine proteases PPACK MI SELEX 228–238 24157838 UltraSynBERT_System_Ranking_for_MI.pkl RNA-target interaction Proteins TAR DNA binding protein 43 TARDBP HTR-SELEX 109 32703884 UltraSynBERT_System_Ranking_for_TARDBP.pkl RNA-target interaction Proteins Ribosomal protein S15 S15 SELEX 87 28587636 UltraSynBERT_System_Ranking_for_S15.pkl RNA-target interaction Proteins RNA-binding motif protein 24 RBM24 HTR-SELEX 109 32703884 UltraSynBERT_System_Ranking_for_RBM24.pkl RNA-target interaction Proteins HIV-1 reverse transcriptase RT SELEX 115–121 23385524 UltraSynBERT_System_Ranking_for_RT.pkl RNA-target interaction Multi(cellular) molecules Triple-negative breast cancer cells TNBC Cell-SELEX 84 32222697 UltraSynBERT_System_Ranking_for_TNBC.pkl RNA-target interaction Multi(cellular) molecules Chinese hamster ovary K1 cells CHO-K1 Cell-SELEX 94–99 29982617 UltraSynBERT_System_Ranking_for_CHO-K1.pkl RNA-target interaction Multi(cellular) molecules Myeloid-derived suppressor cells MDSC Cell-SELEX 74–78 32554710 UltraSynBERT_System_Ranking_for_MDSC.pkl RNA-target interaction Multi(cellular) molecules Human islets ISLETS Tissue-SELEX 80–84 35383192 UltraSynBERT_System_Ranking_for_ISLETS.pkl Human 3′-UTR tissue specificity Human tissue transcriptome Tissue-specific 3′-UTR usage human_22_tissue_three_terminal_UTR.csv Transcriptome-wide split 100 nt 3′-UTR segments APASdb UltraSynBERT_Tissue_Specific_3UTR.pkl Human 3′-UTR tissue specificity Human tissue transcriptome Tissue-specific 3′-UTR usage human_22_tissue_three_terminal_UTR_gene_split.csv Gene-disjoint split 100 nt 3′-UTR segments APASdb UltraSynBERT_Tissue_Specific_3UTR_gene_split.pkl NOTE: The datasets for downstream fine-tuning are available in the Dataset Availability\2. Dataset for fine-tuning folder. The corresponding fine-tuned models are available in the Model Availability\3. Fine-tuned models folder. All fine-tuned checkpoints were initialized from the base UltraSynBERT model.