Replication Package for 'Towards Reliable LLM-Assisted Infrastructure as Code: A Mixed-Methods Study of Generation, Evaluation, and Security'
收藏资源简介:
Complete replication package for the mixed-methods systematic mapping and multi-LLM empirical security study on LLM-assisted Infrastructure as Code (IaC). Contents: QGS-validated database search strings; two-round PRISMA 2020 screening artefacts, including independent dual-screening of the candidate pool (Cohen's kappa = 0.92 over 29 candidates); an extraction codebook and CSV for 31 included primary studies; 116 hand-crafted IaC prompts spanning Terraform, CloudFormation, AWS CDK, Azure Bicep, GCP Deployment Manager, Kubernetes YAML, Helm, and Ansible across AWS, Azure, GCP, Kubernetes, and Linux targets; raw and cleaned outputs from seven LLMs (GPT-5, Claude Opus 4.7, Gemini 2.5 Pro, Grok 4.3, Llama 3.3 70B, Qwen3-Coder 480B, DeepSeek-V4-Flash) with full per-call provenance, totalling 2,436 generations; normalized JSON from five static-analysis scanners (Checkov, tfsec, KICS, Terrascan, Trivy) totalling 19,727 findings; a 202-finding stratified manual-validation sample (population-weighted scanner false-positive rate 45.3 percent) and a 50-artefact scanner false-negative screen; and Python analysis scripts computing Wilson 95 percent confidence intervals, Holm-corrected Fisher pairwise tests, a prompt-clustered GEE logistic model, prompt-cluster bootstrap severity-weighted scores, and pairwise Cohen's kappa across scanners. This archive accompanies a companion research paper currently under peer review.



