CrackedShell-Sec-171

# CrackedShell-Sec-171 CrackedShell-Sec-171 is a file-native benchmark corpus for evaluating agent security behavior, runtime safeguards, tool-use controls, and red-team robustness. The dataset ships each benchmark as a standalone bundle so evaluators can preserve the original multi-file layout during ingestion and execution. ## Highlights - 171 benchmark bundles with per-case metadata, labels, and supporting assets. - Multi-file bundle format designed for agent-skill ingestion, runtime-policy testing, and adversarial evaluation pipelines. - Structured annotations exposed through `manifest.json`, `labels.json`, and optional `file_graph.json` files. - Broad scenario coverage across families such as benign (30), unknown (19), data_exfiltration (18), prompt_injection (17), credential_theft (16), agent_hijacking (15). - Severity-style annotations present in the corpus with values including critical (63), high (57), none (33), medium (17), low (1). ## Repository layout ```text . ├── README.md ├── LICENSE └── skill_bundles/ ├── index.json ├── BND-*/ │ ├── manifest.json │ ├── labels.json │ ├── file_graph.json │ └── files/ │ ├── SKILL.md │ ├── docs/ │ ├── runtime/ │ └── scripts/ ``` ## Data format - `skill_bundles/index.json`: corpus-level index for the full benchmark set. - `manifest.json`: case metadata such as family, severity, permissions, entrypoints, and benchmark annotations. - `labels.json`: per-bundle labels for evaluation and filtering. - `file_graph.json`: optional graph representation of relationships between files inside a bundle. - `files/`: benchmark payload content, including instructions, scripts, runtime configs, prompts, and documentation assets. ## Recommended use This corpus is well suited for: - agent security benchmarking - policy and guardrail evaluation - adversarial skill-ingestion tests - tool permission and runtime-control validation - benchmark-driven red-team experiments ## Citation If you use **CrackedShell-Sec-171** in your work, please cite: ```bibtex @misc{shiroyang2026skillssecurity, author = {Shiro Yang and Rio Yang and Andrew Chen and Pony Ma and {Mind Lab}}, title = {OpenClaw Skill Security: Intent-Capability-Behavior Consistency as a New Framework}, year = {2026}, howpublished = {Mind Lab: A Lab for Experiential Intelligence}, note = {https://macaron.im/mindlab/research/openclaw-skill-security-intent-capability-behavior-consistency-framework} } ``` ## License This dataset is released under the Apache License 2.0. You may use, modify, and redistribute the corpus under the terms of the license; see the `LICENSE` file for the full text. Because some benchmark bundles include executable scripts and adversarial test content, they should only be inspected or run in isolated evaluation environments with appropriate safeguards.