entfane/toxic_chat

---
task_categories:
- text-classification
language:
- en
size_categories:
- 100K<n<1M
---
# Dataset Description
This dataset is a filtered and restructured collection of toxic and sensitive content, specifically curated for training safety classifiers and content moderation models. It combines synthetic data from ToxiGen with real-world samples from ToxicDataset, specifically downsampled and cleaned for high-density training.

## Data Sources
* **[ToxiGen (Train Split)](https://huggingface.co/datasets/toxigen/toxigen-data):** A large-scale synthetic dataset of toxic and benign statements about 13 minority groups.
* **[AiActivity/ToxicDataset](https://huggingface.co/datasets/AiActivity/ToxicDataset) (10% Sample):** A diverse collection of toxic comments, randomly downsampled to 10% to balance the distribution.

## Preprocessing & Cleaning
To ensure the data is "machine-ready" and consistent across different sources, the following transformations were applied:

Prefix Removal: Stripped leading bullet points (- ) from the beginning of every sentence to normalize text structure.

Escape Sequence Correction: Converted literal \\n- string sequences into standard newline characters and removed redundant leading dashes.

Column Pruning: Removed all auxiliary metadata columns, retaining only the core text and label columns (the last 3 columns of the original merged set).

Sampling & Merging: * Full train split of ToxiGen.

0.1 (10%) random sample of AiActivity/ToxicDataset.

## Shuffling: The final combined dataset was globally shuffled with a fixed seed (42) to ensure a diverse distribution of toxicity types across batches.

## Intended Use
Primary Use: Training and fine-tuning LLM safety guardrails.

Secondary Use: Benchmarking moderation APIs against historical and synthetic tropes.

## [!CAUTION]

Warning: This dataset contains highly offensive, biased, and dehumanizing language. It is intended strictly for research and development of safety tools. Use with professional discretion.

## Dataset Schema
The final dataset contains the following 3 columns:
| Column Name | Type | Description |
| :--- | :--- | :--- |
| input | string | The cleaned input |
| output | string | Output |
| label | int | Toxicity classification (0: Neutral, 1: Toxic). |

---
任务类别：
- 文本分类
语言：
- 英语
规模类别：
- 10万 < n < 100万
---
# 数据集说明
本数据集为经筛选与重构的有害与敏感内容集合，专为训练安全分类器与内容审核模型打造。其整合了来自ToxiGen的合成数据，以及来自ToxicDataset的真实样本，并针对高密度训练需求进行了下采样与清洗处理。

## 数据来源
* **ToxiGen（训练拆分集）**：针对13个少数群体的大规模有害与中性陈述合成数据集，链接：https://huggingface.co/datasets/toxigen/toxigen-data
* **[AiActivity/ToxicDataset](https://huggingface.co/datasets/AiActivity/ToxicDataset)（10%采样子集）**：多样化的有害评论集合，经随机下采样至10%以平衡数据分布。

## 预处理与清洗
为确保数据具备"机器可用"属性并统一多源数据格式，我们执行了以下操作：
1. **前缀去除**：移除每条文本开头的项目符号（`- `），以标准化文本结构。
2. **转义序列修正**：将字面量`\n-`字符串序列转换为标准换行符，并移除冗余的前置短横线。
3. **列裁剪**：移除所有辅助元数据列，仅保留核心文本与标签列（原始合并数据集的最后3列）。
4. **采样与合并**：
- 完整保留ToxiGen的训练拆分集
- 随机采样AiActivity/ToxicDataset的10%子集

## 数据洗牌
最终合并后的数据集使用固定随机种子（42）进行全局洗牌，以确保各批次内的有害内容类型分布均匀多样。

## 预期用途
主要用途：训练与微调大语言模型（Large Language Model，LLM）安全护栏。
次要用途：针对历史与合成有害模板，对内容审核API进行基准测试。

## ⚠️ 警告
本数据集包含极具冒犯性、偏见性且非人性化的语言。其仅用于安全工具的研发工作，请谨慎使用。

## 数据集结构
最终数据集包含以下3列：
| 列名 | 数据类型 | 描述 |
| :--- | :--- | :--- |
| input | 字符串 | 清洗后的输入文本 |
| output | 字符串 | 输出结果 |
| label | 整数 | 有害性分类标签（0：中性，1：有害）。