Towards Mitigating Behavioral Drift in LLM-Based Code Refactoring

Towards Mitigating Behavioral Drift in LLM-Based Code Refactoring This artifact accompanies the paper Towards Mitigating Behavioral Drift in LLM-Based Code Refactoring.It contains the datasets, prompts, scripts, supplementary materials, and experimental results used in the study. Repository Structure Behavioral_Drift_Project_Replication/ ├── README.md ├── requirements.txt ├── scripts/ │ └── eq_dfuzz.py │ └── refactor_code.py ├── data/ │ ├── apps.csv │ └── mbpp.csv ├── prompts/ │ ├── baseline.txt │ ├── preserver-r.txt │ └── preserver-m.txt ├── documents/ │ └── annotation_instruction.pdf └── results/ ├── RQ1/ │ └── github_issue_annotation.csv │ ├── RQ2/ │ ├── alomar/ │ │ ├── eq-apps-refactor-alomar-*.csv │ │ └── eq-mbpp-refactor-alomar-*.csv │ │ │ ├── preserver-m/ │ │ ├── eq-apps-refactor-preserver-m-*.csv │ │ └── eq-mbpp-refactor-preserver-m-*.csv │ │ │ └── preserver-r/ │ ├── eq-apps-refactor-preserver-r-*.csv │ └── eq-mbpp-refactor-preserver-r-*.csv │ └── RQ3/ ├── baseline_combined_short.json ├── preserver_m_combined_short.json └── preserver_r_combined_short.json Asterisks (*) replace the names of the evaluated LLMs. Contents data/ # Benchmark datasets used in the experiments. apps.csv mbpp.csv prompts/ # Prompt templates used in the refactoring experiments. baseline.txt # Baseline refactoring prompt preserver-r.txt # PRESERVER-R prompt preserver-m.txt # PRESERVER-M prompt documents/ annotation_instruction.pdf # Instructions for annotating GitHub issues using the open coding method scripts/ eq_dfuzz.py # Implementation of Eq@DFuzz, a functional equivalence checker with fuzzing framework refactor_code.py # Script to obtain refactored code from Ollama and OpenAI results/ RQ1: GitHub issue annotation dataset. RQ2: Refactoring evaluation results for MBPP and APPS across different prompting strategies. RQ3: JSON outputs generated by the baseline and the two prompting strategies on the SWE-Refactor benchmark dataset for Extract Method and Inline Method refactors. Installation Install the required dependencies: pip install -r requirements.txt Getting Refactor Replace the prompt_template field in refactor_code.py with a custom prompt. Set up Ollama or add an OpenAI API key in .env. Run refactor_code.py. python refactor_code.py <input_csv> <openai|ollama> <model_name> <prompt_type> Running Eq@DFuzz To view all available command-line options: python eq_dfuzz.py --help The input CSV must contain at least the following columns: code, refactored_code, primary_method, seed_test_list Additional Resources The user study response spreadsheet is available here.