"Security Analysis of LLM-Generated Terraform Configurations: Prompt Engineering Effects Across Seven Language Models"

"This dataset supports the IEEE Software article \"Why LLMs Fail at Infrastructure Code (and What to Do About It).\" It contains a prompt A\/B security experiment comparing baseline versus security-explicit prompts across seven large language models from four vendors: Claude Sonnet 4.6 (Anthropic), GPT-5.4, GPT-4o, GPT-4.1, and GPT-4o-mini (OpenAI), Llama 4 Scout (Meta), and Mistral Devstral (Mistral). Each model generated an AWS S3 bucket Terraform configuration under two prompt conditions: (A) no security instructions and (B) explicit security requirements including encryption, public access blocking, access logging, lifecycle policies, and versioning. All 14 generated Terraform files were scanned with Checkov v3.2.508 and tfsec v1.28.14 using default rulesets. Key finding: no model produced fully secure output under either prompt condition. Explicit security prompts reduced unique failing Checkov rules for only one of seven models (Claude Sonnet 4.6, from 7 to 4 rules). All models were invoked at temperature 0.0 for deterministic output. The dataset enables independent replication and extension."