Bug Whisperer: Unleashing LLMs to Craft Context-Aware Tests Guided by Path Constraints and Bug Clues

Introduction This is the replication package for the paper "Bug Whisperer: Unleashing LLMs to Craft Context-Aware Tests Guided by Path Constraints and Bug Clues". Task Definition Given the focal method, this task aims to explore the bug detection performance of different prompt strategy in LLMs-based test generation. This replication repository includes experimental datasets and scripts to compare the test generation and bug detection performances of different prompt strategy on nine LLMs. File Information Description dataset.zip: This folder contains the dataset, which includes the validation dataset. The validation dataset file is trigger_bug_fm_all_projects_d4j_processed_503_new.csv from the Defects4J benchmark. evaluation.zip: This folder contains performance evaluation scripts, including CodeBLEU, syntactic correctness rate, compilation passing rate, line coverage, branch coverage, and the number of detected bugs. parser.zip: This folder contains the tree-sitter tool. readme.md: This folder includes the detailed information of dataset and script steps to conduct the experiment. result.zip: This folder includes the experimental results. script.zip: This folder includes the scripts of proposed automated path selection algorithm, LLM inference script, and the experiment script.