Data Availability Artifact for submission #109 Detecting Malicious Data Contamination in Code Generation Benchmarks via Metamorphic Testing

# Data Availability Artifact An anonymous artifact containing code, prompts, benchmark splits, and scripts for reproducing all results is available. The artifact will remain publicly available upon publication. ## Directory Structure This artifact is organized into several phases to facilitate reproducibility. The overall file hierarchy up to the second level is as follows: Data_Availability/ ├── 1_Rewriting_Scripts/ # Original scripts for generating evasion/attacks and rewriting benchmarks (Phase 1) │ └── rewrite/ # Contains Python and bash scripts for text/logic rewriting (MR1) ├── 2_Training_Scripts/ # Core fine-tuning scripts for the controlled contamination experiments (Phase 2) │ ├── LLM_Finetune/ # Execution scripts for full-parameter or different fine-tuning strategies │ └── LoraFinetune/ # Parameter-efficient fine-tuning configurations (LoRA/QLoRA) and data prep ├── 3_Evaluation_Scripts/ # All evaluation and metric calculation scripts (Phase 4) │ # Evaluators such as evaluate_detector.py, compute_mr_metrics*.py, etc. ├── 4_MR_Probes_and_Generated_Files/ # Data hub: Clean/poisoned training sets, MR-S/MR-I scripts, and their datasets (Phase 3) │ ├── counterfactual_detection/ # Constraints file (constraints.py) required for MR derivations │ ├── generate_mr2_impl.py # Probe generation script for Implementation Strategy (MR-I) │ ├── generate_mr2_sem.py # Probe generation script for Semantic Variant (MR-S) │ └── *.jsonl etc. # Core datasets for download (e.g., test_seen.jsonl, mr1_rewritten_full.jsonl, poison_data_*.jsonl) └── 5_Post_Training_Outputs/ # Result hub: Final model outputs and evaluated metric reports ├── LLM_Finetune/ # Result data for full fine-tuning tests ├── LoraFinetune/ # In-depth per-problem results and JSONL logs for the 60+ controlled fine-tuned models └── summary_all_new_lora.csv # Aggregated evaluation metrics (e.g., global AUROC) for all models