Gosai et al. (2025) Synthetic CREs (51k) for GAME API

This Evaluator requests expression predictions for 51,000 synthetic sequences in K562 (erythroid precursors), HepG2 (hepatocytes) and SK-N-SH (neuroblastoma) cells. It then computes the log2 fold change (log2FC) between alternate and reference sequence predictions, and evaluates performance by calculating the Pearson correlation between the measured and predicted log2FC values. Included in Gosai_2024_Synthetic_Evaluator.sif: Scripts to process the data and connect to predictors in the GAME API Scripts to parse the returned predictions and calculate performance metrics All software dependencies /evaluator_data folder contents: 41586_2024_8070_MOESM14_ESM.txt: Sequence files and corresponding measured log2fc values. Total 77,157 sequences but only 51,000 synthetic sequences are used. 41586_2024_8070_MOESM6_ESM.xlsx: Details about the sequences and their design How to run: apptainer run --containall -B /path_to/evaluator_data/:/evaluator_data -B /path_to/prediction_folder/:/predictions Gosai_2024_Synthetic_Evaluator.sif HOST PORT /predictions Notes: The main evaluator script (gosai_evaluator_synthetic.py)will read in the 41586_2024_8070_MOESM14_ESM.txt file and only keep sequences with ["Simulated_Annealing", "FastSeqProp" , "AdaLead"] in the origin column to represent synthetically designed sequences as described in the original publication. Performance metrics are only computed for the 50,748 synthetic predictions (252 sequences do not have experimental measurments) This Evaluator can be used to assess model performance while minimizing sequence leakage in models trained on genomic data Additional information can be found on GitHub: Genomic API for Model Evaluation Original Publication can be found here: Gosai et al. (2025)