KAUGE Benchmark: A Knowledge–Actuation–Gap Dataset for Evaluating AI Secure Code Generation

KAUGE is a benchmark for evaluating how large language models and coding agents handle secure code generation, organized around three layers — Knowledge, Actuation, and the Gap between them. This record contains only the benchmark data; the runnable evaluation harness is maintained at https://github.com/rupampatir/SoK_KAUGE. Contents: questions/ — the L1 (Knowledge) question bank: fixed-answer questions over CERT C, CERT C++, and OWASP Secure Coding Practices. Each item records the rule it tests, a cognitive dimension, an NLP task format, the prompt, and the gold reference answer. exploit_maps/ — used by L2 (generation) and L3 (judge): per-task Secure Coding Practice maps (`cweval_scp_map_dnf.json`, `baxbench_scp_map_dnf.json`), the per-exploit defense formulas in disjunctive normal form (`defense_dnf_map.json`), and one record per exploit with its task spec and security test (`exploits.jsonl`). rules/ — `OWASP_SCP_normalized.json`, the OWASP Secure Coding Practices catalog (id → title + description) used by the L3 judge. agent_manifests/ — JSONL task manifests for the agentic L2 path (coding agents such as codex, claude_code, openhands).