richardyoung/tempest-replication

---
license: mit
task_categories:
- text-classification
language:
- en
tags:
- adversarial-attacks
- llm-safety
- jailbreaks
- red-teaming
size_categories:
- 1K<n<10K
---

# TEMPEST Replication Dataset

Multi-turn adversarial attack results on 10 frontier LLMs.

## Dataset Description

This dataset contains results from replicating the TEMPEST multi-turn jailbreak framework across 10 frontier language models, each evaluated on 100 harmful behaviors from JailbreakBench.

## Key Findings

- **ASR range: 42-100%** - All models vulnerable
- **No scale-safety correlation** (r=-0.12)
- **Thinking mode helps**: Kimi K2 Thinking (42%) vs standard (97%)

## Files

- `data/*.json` - One file per model with 100 behavior evaluations

## Citation

```bibtex
@article{young2025tempest,
title={Replication and Comparative Analysis of Multi-Turn Adversarial Attacks on Frontier Large Language Models},
author={Young, Richard},
year={2025},
institution={University of Nevada Las Vegas}
}
```

## Links

- Paper & Code: [github.com/ricyoung/tempest-replication](https://github.com/ricyoung/tempest-replication)

---
许可证：MIT协议
任务类别：
- 文本分类
语言：
- 英语
标签：
- 对抗攻击
- 大语言模型安全
- 越狱攻击
- 红队测试
样本规模区间：
- 1000<n<10000
---

# TEMPEST复现数据集

针对10款前沿大语言模型（Large Language Model, LLM）的多轮对抗攻击结果。

## 数据集概述

本数据集包含在10款前沿大语言模型上复现TEMPEST多轮越狱攻击框架的实验结果，所有模型均基于JailbreakBench中的100条有害行为完成评估。

## 核心发现

- **攻击成功率（Attack Success Rate, ASR）区间：42%~100%** —— 所有模型均存在安全脆弱性
- **模型规模与安全性无显著相关性**（相关系数r=-0.12）
- **思维模式可提升安全性**：Kimi K2思维模式下攻击成功率为42%，而标准模式下攻击成功率高达97%

## 数据集文件

- `data/*.json`：每个模型对应一个JSON文件，包含该模型在100条有害行为上的评估结果

## 引用格式

bibtex
@article{young2025tempest,
title={Replication and Comparative Analysis of Multi-Turn Adversarial Attacks on Frontier Large Language Models},
author={Young, Richard},
year={2025},
institution={University of Nevada Las Vegas}
}

## 相关链接

- 论文与代码：[github.com/ricyoung/tempest-replication](https://github.com/ricyoung/tempest-replication)