Bug Whisperer: Unleashing LLMs to Craft Context-Aware Tests Guided by Path Constraints and Bug Clues

Introduction This is the replication package for the paper "Bug Whisperer: Unleashing LLMs to Craft Context-Aware Tests Guided by Path Constraints and Bug Clues". Task Definition Given the focal method, this task aims to explore the bug detection performance of different prompt strategies in LLM-based test generation. This replication repository includes experimental datasets and scripts to compare the test generation and bug detection performances of different prompt strategies on nine LLMs. File Information Description dataset.zip: This folder contains the dataset, which includes the validation dataset. The validation dataset file is trigger_bug_fm_all_projects_d4j_processed_503_new.csv from the Defects4J benchmark. evaluation.zip: This folder contains performance evaluation scripts, including CodeBLEU, syntactic correctness rate, compilation passing rate, line coverage, branch coverage, and the number of detected bugs. parser.zip: This folder contains the tree-sitter tool. readme.md: This folder includes the detailed information of the dataset and script steps to conduct the experiment. script.zip: This folder includes the scripts of the proposed automated path selection algorithm, LLM inference script, and the experiment script.