CloneForge: Type-IV Clone Generation Pipeline (implementation and dataset)

CloneForge: Type-IV Clone Generation Pipeline

A hybrid pipeline for generating Type-IV (semantic) code clones using Large Language Models (LLMs) and deterministic validation.

The README.md inside CloneForge.zip contains all the details for each step, including examples, configuration, and workflow.

Overview

CloneForge is a fully automated pipeline designed to generate behaviorally equivalent but syntactically diverse implementations of programs (Type-IV clones).

Unlike naive LLM-based generation, CloneForge enforces:

Semantic correctness via test execution

Syntactic diversity via CodeBLEU filtering

Non-redundancy via clustering

This makes it suitable for:

Training ML-based clone detectors

Benchmark creation

Program transformation studies

Pipeline Architecture

The pipeline consists of six main stages:

Normalization → Generation → Syntactic Filtering → Testing → Repairing → Clustering

Each stage is modular and can be executed independently.

Detailed Pipeline

The pipeline is organized into six steps. Each step is documented in detail in the README.md file:

Step 1: NormalizationStandardizes input data into a unified format, extracting code, tests, metadata, and generating AST representations.

Step 2: Clone GenerationGenerates diverse candidate implementations using configurable LLM prompting strategies and refactorings.

Step 3: Syntactic FilteringRemoves clones that are too similar to the original using a CodeBLEU-based similarity threshold.

Step 4: Semantic TestingExecutes unit tests to ensure behavioral equivalence between generated clones and the original implementation.

Step 5: Repairing Clone CandidatesAttempts to fix partially correct clones using LLM-based re-prompting and test feedback.

Step 6: Representative SelectionApplies clustering to select a diverse, non-redundant subset of valid clones.

Usage

Provide an input dataset with at least a set of test cases.

Specify at least one LLM to be prompted.

Set syntactic and semantic thresholds.

Choose prompt configurations.

The pipeline is implemented in Python and is fully automated.