Dataset and Analysis Scripts for ESEM 2026 Submission

Replication Package for Anonymous ESEM 2026 Submission Overview This repository contains the replication package associated with an anonymous submission to ESEM 2026. The package includes the datasets, extraction procedures, prompts, parsing scripts, and evaluation materials used in the study. The objective of this package is to support transparency, reproducibility, and independent verification of the reported results. Repository Contents Data Extraction Procedures General Extractor_.docx General extraction procedure used during data collection and processing. Astropy Extract.docx Extraction procedure and results associated with the Astropy project. Matplot Extract.docx Extraction procedure and results associated with the Matplotlib project. Django Extract Code.docx Extraction procedure and results associated with the Django project. Swebench Extract.docx Extraction procedure and results associated with the SWE-bench dataset. Ground Truth Materials Ground Truth (Files).docx Ground-truth information related to file-level identification and validation. Ground Truth (Methods).docx Ground-truth information related to method-level identification and validation. Processing and Analysis Parser.docx Description of the parsing procedure used to process extracted information. Prompts_.docx Prompts employed during the experimental procedure. Evaluation F1 Score Calculator.docx Description of the evaluation procedure and F1-score computation methodology. Reproducing the Study To reproduce the reported results: Review the extraction procedures contained in the extractor documents. Apply the parsing process described in Parser.docx. Use the prompts provided in Prompts_.docx during execution. Compare extracted results against the ground-truth datasets. Compute evaluation metrics following the procedure documented in F1 Score Calculator.docx. Anonymity Notice This replication package has been prepared to comply with the double-anonymous review requirements of ESEM 2026. Any identifying information related to authors, affiliations, institutions, or research groups has been intentionally omitted. License The materials in this repository are provided exclusively for research, evaluation, and replication purposes.