malinois-mpra-regression

# Malinois/Gosai MPRA Regression This dataset preprocesses the Gosai et al. 2024 supplementary MPRA table used by the Malinois benchmark for supervised DNA-to-activity regression. Each row contains a DNA sequence and three cell-type-specific activity targets: `K562_log2FC`, `HepG2_log2FC`, and `SKNSH_log2FC`. No new license is asserted by this preprocessing. Users should follow the terms of the source publication and supplementary data. ## Source - Publication: Gosai et al., *Machine-guided design of cell-type-targeting cis-regulatory elements*, Nature 2024. - Source table: `41586_2024_8070_MOESM4_ESM.txt`. - Source URL: `https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-024-08070-z/MediaObjects/41586_2024_8070_MOESM4_ESM.txt`. ## Splits Chromosome splits match the Carbon fine-tuning experiments and the public Malinois setup we used: | Split | Chromosomes | Rows | Rows with all SE <= 1.0 | |---|---:|---:|---:| | train | all except validation/test chromosomes | 668,946 | 627,661 | | validation | 19, 21, X | 62,406 | 58,811 | | test | 7, 13 | 66,712 | 62,582 | Total rows after filtering finite targets/standard errors and nonempty sequences: 798,064. ## Columns - `id`: original row identifier from the source table. - `split`: train, validation, or test. - `chromosome`: normalized chromosome label. - `data_project`, `oligo`, `variant_class`: source metadata. - `sequence`: uppercase DNA sequence. - `reverse_complement`: reverse complement of `sequence`. - `forward_rc_concat`: `<dna>sequence</dna><dna>reverse_complement</dna>`, matching the best Carbon fine-tuning recipe. - `K562_log2FC`, `HepG2_log2FC`, `SKNSH_log2FC`: raw regression targets. - `K562_lfcSE`, `HepG2_lfcSE`, `SKNSH_lfcSE`: target standard errors. - `*_train_zscore`: target standardized using train-split mean/std. - `all_se_le_1`: true when all three SE columns are `<= 1.0`; this was the main reported validation/test metric filter. - `any_log2fc_gt_0_5`: true when any target is greater than `0.5`; this was used for optional high-activity training upsampling. Train z-score statistics: | Target | Mean | Std | |---|---:|---:| | K562_log2FC | 0.49943020 | 1.17725282 | | HepG2_log2FC | 0.46267671 | 1.05124023 | | SKNSH_log2FC | 0.41405871 | 1.16609108 | ## Usage ```py from datasets import load_dataset ds = load_dataset("HuggingFaceBio/malinois-mpra-regression") train = ds["train"] validation_metric = ds["validation"].filter(lambda row: row["all_se_le_1"]) example = train[0] sequence = example["forward_rc_concat"] labels = [ example["K562_log2FC_train_zscore"], example["HepG2_log2FC_train_zscore"], example["SKNSH_log2FC_train_zscore"], ] ``` To recreate the dataset: ```sh python create_dataset.py --repo-id HuggingFaceBio/malinois-mpra-regression --push ```