Virtual Machines for DeepRed: Partial-Credit Evaluation of LLM Agents in CTFs

<strong>DeepRed Challenges</strong><br>This dataset contains Capture The Flag (CTF) challenge VirtualBox images used for benchmarking Large Language Models in penetration testing scenarios. The challenges range cover various cybersecurity domains including web exploitation, privilege escalation, network enumeration, and more. This dataset is developed to be used in the DeepRed benchmark harness: https://github.com/AISE-TUDelft/DeepRed-LLMAgent<br><strong>Challenge Attribution</strong><br>All of these challenges are sourced from HackMyVM (https://hackmyvm.eu/), a platform providing free CTF challenges for cybersecurity learning and practice. <br><strong>Paper</strong>This dataset is published as part of our AIware 2026 paper <em>Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges</em>. Please refer to the preprint (https://arxiv.org/abs/2604.19354) or paper (https://doi.org/10.1145/3805760.3814926) for more details.