Towards Human-interpretable Explanation in Code Clone Detection using LLM-based Post Hoc Explainer

Overview This artifact accompanies the APSEC 2025 Research Track submission on Towards Human-interpretable Explanation in Code Clone Detection using LLM-based Post Hoc Explainer. The research focuses on using Large Language Models (LLMs) with local neighborhood sampling with in-context learning to provide explanations for GraphCodeBERT's code clone detection predictions. Artifact Contents 1. Core Implementation GraphCodeBERT_CloneDetection.ipynb: Complete Jupyter notebook containing: GraphCodeBERT model setup and training Code clone detection experiments on BigCloneBench and Google Code Jam datasets In-Context Learning prompt generation for explanation LLM-based explanation pipeline 2. In-Context Learning Prompts (ICL_Prompts) The ICL_Prompts directory contains structured prompts for 10 different Google Code Jam pairs (data_GoogleCodeJam_Pair1 through data_GoogleCodeJam_Pair10) used in the manual validation step, each with varying context lengths: layer_size4.txt: prompts with 4 examples layer_size8.txt: prompts with 8 examples layer_size16.txt: prompts with 16 examples layer_size32.txt: prompts with 32 examples layer_size48.txt: prompts with 48 examples layer_size64.txt: prompts with 64 examples Each prompt file is based on the prompt template structure: Context: Description of GraphCodeBERT model Dataset: Examples with code pairs, confidence scores, and predictions Question: A query asking for explanation of a specific code pair Instructions for generating explanations 3. Validation Results Manual_Validation_Results.xlsx: Manual evaluation results of generated explanations including quality assessments and human annotations. Usage Instructions Prerequisites pip install torch transformers tree_sitter scikit-learn Running the Code Open `GraphCodeBERT_CloneDetection.ipynb` in Jupyter/Colab Follow the notebook cells to: Set up the GraphCodeBERT model Load and preprocess datasets Run clone detection experiments Generate prompts for explanation File Organization ├── GraphCodeBERT_CloneDetection.ipynb # Main implementation ├── Manual_Validation_Results.xlsx # Human evaluation results ├── ICL_Prompts/ # Generated prompts │ ├── data_GoogleCodeJam_Pair1/ # Pair 1 prompts │ │ ├── layer_size4.txt # 4-example prompts │ │ ├── layer_size8.txt # 8-example prompts │ │ └── ... # Other sizes │ ├── data_GoogleCodeJam_Pair2/ # Pair 2 prompts │ └── ... # Additional pairs └── README.md # This file Citation Please cite this artefact using the DOI provided by Zenodo. Contact For questions about this artifact or research, please contact the authors [to be updated due to double-blind policy].