Agarwal et al. (2025) Joint Library Dataset (56k sequences) Evaluator Container for the Genomic API for Model Evaluation (GAME)

This container is the Evaluator configured for the Genomic API for Model Evaluation (GAME), designed specifically to evaluate model predictions against the Agarwal et al. (2025) Joint Library dataset. The dataset consists of 56,982 candidate cis-regulatory elements (cCREs), including enhancers and promoters systematically tested across: induced pluripotent stem cells (iPS cells; WTC11) lymphoblasts (K562) human hepatocytes (HepG2) The dataset also includes positive and negative control sequences. Evaluation Workflow Expression Prediction Evaluates Predictor performance for expression (point predictions). Computes the Pearson correlation between measured values (log2(rna/dna)) and predicted values. Cell Type Specificity Evaluates cell type specificity between measured and predicted values by calculating the pearson correlation between: [cell1_measured - cell2_measured] and [cell1_predicted - cell2_predicted]. Included in agarwal_joint_56k.sif: Scripts to process the data and connect to predictors in the GAME API Scripts to parse the returned predictions and calculate performance metrics All software dependencies /evaluator_data folder contents: 2023-03-03628C-Table_S10-joint_lib_design_56k_measured.xlsx: Sequence design and coordinate information (each sequence is a 230-bp oligonucleotide) all_cell_type_measured.xlsx: Measurments for the ~56k sequences across all 3 cell types [log2(rna/dna)] How to run: apptainer run --containall -B /path_to/evaluator_data/:/evaluator_data -B /path_to/prediction_folder/:/predictions agarwal_joint_56k.sif HOST PORT /predictions Notes: Details about QC filtering can be found in the original publication The original 2023-03-03628C-Table_S10-joint_lib_design.xlsx sequence file from the paper was filtered to only keep sequences with measured values in all_cell_type_measured.xlsx When evaluating expression predictions for each cell type, the number of measurements used for Pearson correlation may vary due to filtering. Cell type specificity evaluation uses only sequences with measurements across all three cell types to ensure fairness Additional information can be found on GitHub: Genomic API for Model Evaluation Original publication can be found here: Agarwal et al. 2025