CloneForge: Type-IV Clone Generation Pipeline (implementation and dataset)

CloneForge: Type-IV Clone Generation Pipeline

A hybrid pipeline for generating Type-IV (semantic) code clones using Large Language Models (LLMs) and deterministic validation.

The README.md inside CloneForge.zip contains all the details for each step, including examples, configuration, and workflow.

Overview

CloneForge is a fully automated pipeline designed to generate behaviorally equivalent but syntactically diverse implementations of programs (Type-IV clones).

Unlike naive LLM-based generation, CloneForge enforces:

Semantic correctness via test execution

Syntactic diversity via CodeBLEU filtering

Non-redundancy via clustering

This makes it suitable for:

Training ML-based clone detectors

Benchmark creation

Program transformation studies

Pipeline Architecture

The pipeline consists of six main stages:

Normalization → Generation → Syntactic Filtering → Testing → Repairing → Clustering

Each stage is modular and can be executed independently.

Detailed Pipeline

The pipeline is organized into six steps. Each step is documented in detail in the README.md file:

Step 1: NormalizationStandardizes input data into a unified format, extracting code, tests, metadata, and generating AST representations.

Step 2: Clone GenerationGenerates diverse candidate implementations using configurable LLM prompting strategies and refactorings.

Step 3: Syntactic FilteringRemoves clones that are too similar to the original using a CodeBLEU-based similarity threshold.

Step 4: Semantic TestingExecutes unit tests to ensure behavioral equivalence between generated clones and the original implementation.

Step 5: Repairing Clone CandidatesAttempts to fix partially correct clones using LLM-based re-prompting and test feedback.

Step 6: Representative SelectionApplies clustering to select a diverse, non-redundant subset of valid clones.

Usage

Provide an input dataset with at least a set of test cases.

Specify at least one LLM to be prompted.

Set syntactic and semantic thresholds.

Choose prompt configurations.

The pipeline is implemented in Python and is fully automated.

CloneForge：第四类代码克隆生成流水线

本框架为一套混合式流水线，用于借助大语言模型（Large Language Models，LLMs）与确定性验证手段生成第四类（Type-IV）语义型代码克隆。

CloneForge.zip 内置的 "README.md" 文档包含各步骤的全部细节，涵盖示例、配置信息与工作流程。

## 概述
CloneForge 是一套全自动化流水线，旨在生成与原程序行为等价但语法各异的实现版本（即第四类代码克隆）。
与朴素的大语言模型生成方案不同，CloneForge 通过以下三重机制保障生成质量：
- 通过代码测试验证语义正确性
- 基于 CodeBLEU 过滤实现语法多样性
- 通过聚类消除样本冗余
该流水线适用于以下三类场景：
- 训练基于机器学习的代码克隆检测器
- 构建基准测试数据集
- 开展程序转换相关研究

## 流水线架构
本流水线包含六个核心阶段：标准化 → 克隆生成 → 语法过滤 → 测试验证 → 修复优化 → 聚类筛选
各阶段均为模块化设计，可独立执行。

## 详细流水线步骤
本流水线分为六个步骤，各步骤的详细说明均收录于 "README.md" 文档中：
1. **标准化（Normalization）**：将输入数据统一为标准格式，提取代码、测试用例、元数据，并生成抽象语法树（Abstract Syntax Tree，AST）表示。
2. **克隆生成（Clone Generation）**：通过可配置的大语言模型提示策略与重构手段，生成多样化的候选实现版本。
3. **语法过滤（Syntactic Filtering）**：基于 CodeBLEU 相似度阈值，移除与原代码过于相似的克隆样本。
4. **语义测试（Semantic Testing）**：执行单元测试，确保生成的克隆样本与原实现的行为等价。
5. **克隆候选样本修复（Repairing Clone Candidates）**：借助大语言模型的重提示机制与测试反馈，修复部分合格的候选克隆。
6. **代表性样本筛选（Representative Selection）**：通过聚类算法筛选出多样化且无冗余的有效克隆子集。

## 使用方法
1. 提供包含至少一组测试用例的输入数据集
2. 指定至少一个待调用的大语言模型
3. 设置语法与语义阈值
4. 选择提示词配置方案

本流水线基于 Python 实现，支持全自动化运行。