Replication package for the paper: "Attentionsmelling: Using Large Language Models to Identify Code Smells"

Replication Package Context

This is the replication package for the paper "AttentionSmelling — Using Large Language Models to Identify Code Smells". The paper was originally published in the SBES 2025 -39th Brazilian Symposium on Software Engineering.

Experiment

This replication package accompanies the paper "Attentionsmelling: Using Large Language Models to Identify Code Smells," which systematically evaluates GPT-4o’s ability to detect three key code smells (Long Method, God Class, and Feature Envy) in Java code. The artifact centers on a Jupyter notebook that guides users through all steps of data preprocessing, experiment execution, and error analysis, following the paper’s methodology. It uses a refined oracle derived from the Crowdsmelling dataset, applying majority voting to reduce subjectivity and focusing on the jasml-0.10 project for consistency. Four experimental configurations progressively enhance the LLM prompts, incorporating definitions, code metrics, and hyperparameter tuning, leading to a 64% improvement in AUC and 56% in F1-score. The notebook includes instructions, dataset references, and scripts for reproducing the full analysis pipeline. Designed for reproducibility and accessibility, it requires only basic Python and Jupyter knowledge and runs on standard hardware or Google Colab. The package enables detailed evaluation of LLM-based code smell detection, provides error and confidence analysis, and offers a practical resource for both research and education. All artifacts are publicly available, well-documented, and ready for reuse or extension. Limitations include the focus on a single Java project and reliance on the adopted code smell definitions, which may affect generalizability to other contexts.

复现包背景说明
本包为论文《AttentionSmelling——基于大语言模型（Large Language Model）识别代码坏味道（Code Smells）》的复现材料。该论文原发表于第39届巴西软件工程研讨会（SBES 2025）。

实验说明
本复现包配套上述论文，系统评估了GPT-4o在Java代码中检测三类核心代码坏味道：长方法（Long Method）、上帝类（God Class）、特征依恋（Feature Envy）的能力。本复现包的核心为一份Jupyter笔记本，其遵循论文中的研究方法，引导用户完成数据预处理、实验执行与误差分析的全流程。该工件采用从Crowdsmelling数据集提炼得到的优化标注基准，通过多数投票法降低标注主观性，并统一以jasml-0.10项目作为实验对象以保证一致性。

本次实验共设置四种递进式的大语言模型提示词配置，依次融入代码坏味道定义、代码度量指标与超参数调优策略，最终实现曲线下面积（AUC）提升64%、F1分数提升56%的效果。该笔记本包含完整分析流程的复现说明、数据集引用信息与执行脚本。

本包兼顾可复现性与易用性，仅需掌握基础Python与Jupyter操作知识，即可在标准硬件或Google Colab平台上运行。该包可支持基于大语言模型的代码坏味道检测的精细化评估，提供误差与置信度分析工具，同时可为学术研究与教学实践提供实用参考资源。所有研究工件均公开可用，文档完善，可直接复用或拓展。

本包存在一定局限性：仅针对单个Java项目开展实验，且依赖于本次研究采用的代码坏味道定义，这可能会限制其在其他场景下的泛化能力。