Dataset for "Using Explainable Artificial Intelligence for Classifying Roles in Source Code"

This dataset contains the materials used in the study “Using Explainable Artificial Intelligence for Classifying Roles in Source Code.” It includes manual Entity–Boundary–Control (EBC) role annotations, LIME-based explanation validations, and the machine-learning models and notebooks used to generate topic classifications, EBC predictions, and XAI explanations. The package supports reproducibility of the study’s workflow and enables further research on interpretable ML techniques for program comprehension.

The materials support three main components of the study:1. Manual EBC Role AnnotationThese files provide human-labeled ground truth for 64 Java source files across four packages (awt, awt.dnd, awt.text, awt.converter) from the Pumpernickel open-source project.Included materials: • Manual Annotation EBC – awt.pdf • Manual Annotation EBC.xlsxEach file entry includes the package, file name, code snippet, role label (Entity/Boundary/Control), and expert reviewer's annotation notes.

2. XAI-Based EBC Role ValidationThis folder contains expert validation of LIME explanations for 60 test files (20 per role).Included materials: • EBC XAI Validation – Validation.pdf • EBC XAI Validation.xlsx

The validation includes: • LIME feature weights for each prediction • Expert ratings on a 1–5 scale • Additional comments on explanation clarity and role fit

3. Machine Learning Pipelines and ModelsThis section includes the complete ML artifacts used in the study for topic classification and EBC role prediction, ensuring full reproducibility.Included materials: • tfidf_rf_ebc_pipeline.pkl • tfidf_vectorizer_best_lite.pkl • random_forest_model_best_lite.pkl • topic_classification.ipynb • EBC_LIME.ipynb

These files allow direct recreation of: • Topic-model probabilities • File-level EBC predictions • LIME explanation outputs for each source file

Purpose of the DatasetThis dataset enables researchers to: • Study multi-granular program comprehension (package → file → explanation) • Reproduce ML experiments for EBC classification • Replicate and validate XAI (LIME-based) explanations • Explore program comprehension with interpretable machine learning techniques • Build extensions for top-down understanding tools