PatternForge: A Benchmark Dataset for Pattern-Based Refactoring of Code Smells

PatternForge Dataset and Artifacts This repository provides the complete dataset and experimental artifacts used in the study: PatternForge: A Multi-Agent LLM Framework for Refactoring Code Smells into Design Patterns The repository is intended to support transparency, reproducibility, and reuse of the benchmark introduced in the paper. All artifacts required to inspect, reproduce, and extend the experimental results are publicly available. Repository Contents The dataset is organized according to a standardized and documented directory structure.Each instance corresponds to a single dominant code smell, a fixed complexity level, and a specific programming language. dataset/ └── SmellType/ └── ComplexityLevel/ └── InstanceID/ ├── original/ # Original smelly implementation ├── refactored/ # Refactored version generated by PatternForge ├── tests/ # Executable test suite for behavioral validation └── metadata.json # Smell type, target pattern, language, complexity This separation explicitly distinguishes input artifacts from generated outputs, preventing ambiguity between manually authored code and automatically refactored results. Dataset Description The dataset consists of 72 curated instances, evenly distributed across: Six canonical code smellsLong Method, God Class, Feature Envy, Switch Statement, Data Class, Duplicated Code Three complexity tiersEasy, Medium, Hard Two statically typed object-oriented languagesJava and C# Each instance is designed to exhibit one dominant architectural deficiency, following canonical definitions from the refactoring literature (e.g., Fowler, Kerievsky, Olbrich et al.).Overlapping or confounding smells are intentionally avoided to enable controlled, fine-grained architectural analysis. Refactored Implementations All code under the refactored/ directories is generated exclusively by the PatternForge framework using: prompt-guided, multi-agent reasoning, explicit pattern-role planning, bounded validation with compilation and test execution. No manual post-editing or corrective intervention has been applied to the refactored outputs. Behavioral Validation Each dataset instance includes an executable test suite under the tests/ directory.These tests are used to verify behavioral equivalence between the original and refactored implementations. The test suites are intentionally lightweight and instance-specific, serving as a conservative oracle to detect functional deviations introduced during architecture-level refactoring. Metadata Each dataset instance is accompanied by a metadata.json file that records static structural metrics for both the original (smelly) implementation and the refactored version generated by PatternForge. The metadata file is organized into two main sections: original: structural metrics extracted from the initial implementation containing the code smell refactored: structural metrics extracted from the pattern-based refactored implementation For each section, the following metrics are reported: Lines of Code (LOC): number of logical source code lines Class Count: number of declared classes Interface Count: number of declared interfaces Type Count: total number of types (classes + interfaces) Cyclomatic Complexity: control-flow complexity computed using the standard McCabe definition Coupling Between Objects (CBO): number of distinct type-level dependencies This metadata structure enables direct, quantitative comparison of structural properties before and after refactoring, supporting the analysis of complexity variation, architectural expansion, and coupling effects introduced by pattern-based transformations. Moreover, the standardized metadata schema facilitates automated metric aggregation, cross-instance statistical analysis, and reproducibility of experimental configurations, making it suitable for systematic benchmarking of architecture-level refactoring approaches. Intended Use This dataset is designed for: evaluating architecture-level refactoring frameworks, studying pattern-aware automated refactoring, benchmarking LLM-based multi-agent code transformation systems. It is intended as a capability and feasibility benchmark, not as a large-scale industrial corpus. Reviewer-Oriented Note This repository is structured to ensure: clear separation between original and generated artifacts, full inspectability of refactoring outputs, reproducibility of behavioral validation,PatternForge 数据集与配套工件 本仓库提供了研究《PatternForge：一种将代码异味（code smell）重构为设计模式的多智能体大语言模型（LLM）框架》中使用的完整数据集与实验配套工件。 本仓库旨在支持该论文中提出的基准测试的可透明性、可复现性与复用性。所有用于检视、复现并拓展实验结果的配套工件均已公开发布。 ## 仓库内容 数据集采用标准化且附带文档说明的目录结构进行组织。每个数据实例对应单一主导型代码异味、固定复杂度等级与特定编程语言。 目录结构如下： dataset/ └── SmellType/ └── ComplexityLevel/ └── InstanceID/ ├── original/ # 原始带代码异味的实现 ├── refactored/ # PatternForge生成的重构版本 ├── tests/ # 用于行为验证的可执行测试套件 └── metadata.json # 记录异味类型、目标设计模式、编程语言、复杂度的元数据文件 这种目录划分清晰区分了输入工件与生成输出，避免了手动编写代码与自动重构结果之间的歧义。 ## 数据集说明 本数据集共包含72个经过精选的数据实例，均匀分布于以下类别中： 1. 六种典型代码异味：长方法（Long Method）、神类（God Class）、特征依恋（Feature Envy）、Switch语句（Switch Statement）、数据类（Data Class）、重复代码（Duplicated Code） 2. 三个复杂度层级：简单（Easy）、中等（Medium）、困难（Hard） 3. 两种静态类型面向对象编程语言：Java与C# 每个数据实例均旨在呈现单一主导型架构缺陷，遵循重构领域文献（如Fowler、Kerievsky、Olbrich等人的研究）中的标准定义。本数据集刻意避免了重叠或相互干扰的代码异味，以支持受控的细粒度架构分析。 ## 重构实现 所有位于`refactored/`目录下的代码均由PatternForge框架独家生成，其生成过程采用了：提示词引导的多智能体推理、显式模式-角色规划、结合编译与测试执行的有界验证。重构输出未经过任何手动后编辑或修正干预。 ## 行为验证 每个数据实例的`tests/`目录下均包含一套可执行测试套件，用于验证原始实现与重构实现之间的行为等价性。该测试套件刻意设计为轻量级且专属于单个数据实例，作为保守判定标准，用于检测架构级重构过程中引入的功能偏差。 ## 元数据 每个数据实例均附带一个"metadata.json"文件，用于记录原始（带代码异味的）实现与PatternForge生成的重构版本的静态结构指标。元数据文件分为两个主要部分： - `original`：从包含代码异味的初始实现中提取的结构指标 - `refactored`：从基于模式的重构实现中提取的结构指标 每个部分均报告以下指标： 1. 代码行数（Lines of Code, LOC）：逻辑源代码行数 2. 类数量（Class Count）：声明的类的数量 3. 接口数量（Interface Count）：声明的接口的数量 4. 类型总数（Type Count）：类型总数（类+接口） 5. 圈复杂度（Cyclomatic Complexity）：采用标准McCabe定义计算的控制流复杂度 6. 对象间耦合度（Coupling Between Objects, CBO）：不同类型级依赖的数量 该元数据结构支持对重构前后的结构属性进行直接的定量对比，助力分析复杂度变化、架构扩展以及基于模式的转换引入的耦合效应。此外，标准化的元数据架构便于自动指标聚合、跨实例统计分析与实验配置的复现，适用于架构级重构方法的系统性基准测试。 ## 预期用途 本数据集旨在用于： 1. 评估架构级重构框架 2. 研究感知模式的自动化重构技术 3. 基于大语言模型（LLM）的多智能体代码转换系统的基准测试 本数据集作为能力与可行性基准测试集，而非大规模工业语料库。 ## 面向评审者的说明 本仓库的结构设计旨在保障： 1. 原始工件与生成工件的清晰区分 2. 重构输出的完全可检视性 3. 行为验证的可复现性