TELOS Adversarial Validation Dataset

Validation results for TELOS AI governance framework demonstrating 0 observed attack successes across 2,550 adversarial attacks from four evaluation sources (AILuminate, MedSafetyBench, HarmBench, SB 243-aligned suite). Includes full forensic audit trail with JSONL governance event logs. Key Results: - 2,550 attacks validated (1,200 AILuminate + 900 MedSafetyBench + 400 HarmBench + 50 SB 243- aligned) - 0/2,550 observed attack successes - 95% CI upper bound: ~0.15% - 95.8% autonomous blocking (Tier 1) - Six Sigma performance: <2% human escalation - Three-tier governance architecture (Primacy Attractor → RAG → Human) Benchmarks: - AILuminate Standard Benchmark (1,200 prompts, 12 NIST AI RMF harm categories) - MedSafetyBench (900 prompts, NeurIPS 2024) - HarmBench (400 prompts, Center for AI Safety) - SB 243-Aligned Evaluation Suite (50 prompts, internal benchmark aligned with California child safety categories) Files Included: - Complete validation datasets from all four evaluation sources - Statistical analysis summary - Tier distribution data - ERRATA_v1.1.md (validation status clarification) Forensic Audit Trail (v2.0) - harmbench_forensic_forensic_summary.json - HarmBench aggregate statistics - harmbench_forensic_forensic_results.json - HarmBench per-prompt forensic data - harmbench_forensic_fidelity_distribution .csv - HarmBench fidelity scores - harmbench_forensic_governance_report.html - HarmBench interactive visualization - traces/session_harmbench_forensic_*.jsonl - HarmBench JSONL governance event log - medsafetybench_forensic_forensic_summary.json - MedSafetyBench aggregate statistics - medsafetybench_forensic_forensic_results.json - MedSafetyBench per-prompt forensic data - medsafetybench_forensic_fidelity_distribution.csv - MedSafetyBench fidelity scores - medsafetybench_forensic_governance_report.html - MedSafetyBench interactive visualization - traces/session_medsafetybench_forensic_*.jsonl - MedSafetyBench JSONL governance event log Validation Status: This dataset demonstrates proof-of-concept validation of the TELOS governance methodology under ' black-box threat models. The healthcare PA and RAG corpus were constructed from authoritative public domain sources (HIPAA Privacy Rule, HHS guidance, peer-reviewed clinical literature) but have not been formally validated by external healthcare compliance professionals or clinical researchers. Results should be interpreted as methodology demonstration, not certification for clinical deployment. See ERRATA_v1.1.md for details. License: Apache 2.0 Validation Date: Original validation 2024-12-21 (forensic audit added 2026-01-25, AILuminate/SB 243- aligned validation added 2026-01)# TELOS人工智能治理框架验证结果 本数据集针对TELOS人工智能治理框架（TELOS AI governance framework）开展验证，在来自4个评估源的共计2550次对抗性攻击（adversarial attacks）中，未观测到任何攻击成功案例。这4个评估源分别为AILuminate、MedSafetyBench、HarmBench以及SB 243对齐套件（SB 243-aligned suite）。数据集包含完整的取证审计跟踪（forensic audit trail），附带JSONL格式治理事件日志。 ## 核心结果 - 共完成2550次攻击验证（其中AILuminate测试集1200次、MedSafetyBench测试集900次、HarmBench测试集400次、SB 243对齐套件测试集50次） - 2550次攻击均未成功（0/2550） - 95%置信区间（95% Confidence Interval, CI）上限约为0.15% - 自主拦截率达95.8%（一级拦截，Tier 1） - 六西格玛（Six Sigma）性能表现：人工升级率低于2% - 三级治理架构：优先级吸引器（Primacy Attractor）→ 检索增强生成（Retrieval-Augmented Generation, RAG）→ 人工干预 ## 测试基准 - AILuminate标准基准测试集（AILuminate Standard Benchmark）：包含1200条提示词，覆盖12个美国国家标准与技术研究院人工智能风险管理框架（National Institute of Standards and Technology AI Risk Management Framework, NIST AI RMF） harm分类 - MedSafetyBench测试集：包含900条提示词，源自NeurIPS 2024 - HarmBench测试集：包含400条提示词，由人工智能安全中心（Center for AI Safety）发布 - SB 243对齐评估套件（SB 243-Aligned Evaluation Suite）：包含50条提示词，为内部基准测试，与加州儿童安全分类标准对齐 ## 包含文件 - 来自全部4个评估源的完整验证数据集 - 统计分析汇总报告 - 拦截层级分布数据 - ERRATA_v1.1.md：验证状态澄清文档 ## 取证审计跟踪（v2.0） - harmbench_forensic_forensic_summary.json：HarmBench取证汇总文件 - HarmBench聚合统计数据 - harmbench_forensic_forensic_results.json：HarmBench取证结果文件 - HarmBench单条提示词取证数据 - harmbench_forensic_fidelity_distribution.csv：HarmBench保真度评分文件 - harmbench_forensic_governance_report.html：HarmBench交互式可视化治理报告 - traces/session_harmbench_forensic_*.jsonl：HarmBench JSONL格式治理事件日志 - medsafetybench_forensic_forensic_summary.json：MedSafetyBench聚合统计数据 - medsafetybench_forensic_forensic_results.json：MedSafetyBench单条提示词取证数据 - medsafetybench_forensic_fidelity_distribution.csv：MedSafetyBench保真度评分文件 - medsafetybench_forensic_governance_report.html：MedSafetyBench交互式可视化治理报告 - traces/session_medsafetybench_forensic_*.jsonl：MedSafetyBench JSONL格式治理事件日志 ## 验证状态 本数据集展示了TELOS治理方法在黑盒威胁模型（black-box threat models）下的概念验证结果。医疗PA与检索增强生成语料库均源自权威公开领域资源，包括《健康保险流通与责任法案隐私规则（Health Insurance Portability and Accountability Act Privacy Rule, HIPAA）》、美国卫生与公众服务部（Department of Health and Human Services, HHS）指南以及同行评议的临床文献，但尚未经过外部医疗合规专业人员或临床研究人员的正式验证。本数据集结果仅作为治理方法的演示验证，不可作为临床部署的认证依据。详细信息请参阅ERRATA_v1.1.md文档。 ## 许可证 Apache 2.0许可证 ## 验证日期 首次验证时间：2024年12月21日；取证审计跟踪更新时间：2026年1月25日；AILuminate与SB 243对齐套件验证更新时间：2026年1月