Replication package for "Directed Symbolic Execution for Vulnerability Discovery: An LLM-Guided Approach in KLEE"

# Replication Package This repository provides the replication package for "Directed Symbolic Execution for Vulnerability Discovery: An LLM-Guided Approach in KLEE". ### 1. RepositoriesSee all experiment code in `repositories.7z` archive, including: - cgs_evaluation: Code for CGS evaluation- klee-llm: Code for KLEECopilot and SDSE baseline evaluation- docker-empc: Docker image for empc and builtin- learch: Code for learch ### 2. Docker ImagesSee all experiment Docker images in `images.7z` archive, you can also build the Docker images by using repositories in `repositories.7z` archive.- images: Docker images for all experiments, see `images/README.md` for more details. ### 3. Data ArchivesSee all experiment results in `data_archive.7z` archive, including: * Ablation study results: - results_ab0: ablation study 1 (internal comments) - results_ab1: ablation study 2 (internal comments) - results_random: with random markings (different markings) - results_no_marking: without markings (different markings) - results_sanitizer: with sanitizer markings (different markings) - results_sinks: with sink markings (different markings) - results_semgrep: with markings generated by Semgrep static analyzer (different markings) - results_infer: with markings generated by Infer static analyzer (different markings) - results_codeql: with markings generated by CodeQL static analyzer (different markings) - results_dse_deepseek-coder_33b: with SDSE and DeepSeek-Coder-33B (different searcher) - results_deepseek-coder_33b_ne: without negative prompts (prompt ablation) - results_deepseek-coder_33b_np: without positive prompts (prompt ablation) * Baselines: - results_learch: learch evaluation results - results_cgs: CGS evaluation results - results_empc: empc evaluation results - results_all_baselines: evaluation results of all baselines - results_sgs: SGS evaluation results * Different LLMs: - results_deepseek-coder_6.7b: KLEECopilot with DeepSeek-Coder-6.7B evaluation results - results_deepseek-coder_33b: KLEECopilot with DeepSeek-Coder-33B evaluation results - results_qwen2.5-coder_7b: KLEECopilot with Qwen2.5-Coder-7B evaluation results - results_qwen2.5-coder_32b: KLEECopilot with Qwen2.5-Coder-32B evaluation results - results_gpt-4o: KLEECopilot with GPT-4o evaluation results - results_deepseek-v4-flash: KLEECopilot with DeepSeek-v4-Flash evaluation results - results_gemini-2.5-flash: KLEECopilot with Gemini-2.5-Flash evaluation results * Data leakage experiments: - results_add2_cgs: data leakage experiment results for CGS evaluation - results_add2_learch: data leakage experiment results for learch evaluation - results_add2_empc: data leakage experiment results for empc and builtin evaluation - results_add2_llm: data leakage experiment results for KLEECopilot with qwen2.5-coder-32b evaluation results ### 4. Violation Archive* `reported_violations.pdf`: a PDF file that contains all the reported and confirmed violations in our paper.* `confirmed one, fixed one`: which means the one violation is confirmed by the developers and has been fixed.* `confirmed one`: which means the one violation is confirmed by the developers but has not been fixed yet. ### 5. Pre-built Benchmark Programs* `build.tgz`: a tarball that contains all the pre-built benchmark programs for our experiments.* `build_additional.tgz`: a tarball that contains the pre-built benchmark program for data leakage experiments. **Note**1. Warning, those data archives are **very huge to extract**, data_archive.7z will take about 804.78 GB. --- ## Citation If you use this replication package, please cite our paper: ```[Under-review]```