entfane/toxic_chat

---
task_categories:
- text-classification
language:
- en
size_categories:
- 100K<n<1M
---
# Dataset Description
This dataset is a filtered and restructured collection of toxic and sensitive content, specifically curated for training safety classifiers and content moderation models. It combines synthetic data from ToxiGen with real-world samples from ToxicDataset, specifically downsampled and cleaned for high-density training.

## Data Sources
* **[ToxiGen (Train Split)](https://huggingface.co/datasets/toxigen/toxigen-data):** A large-scale synthetic dataset of toxic and benign statements about 13 minority groups.
* **[AiActivity/ToxicDataset](https://huggingface.co/datasets/AiActivity/ToxicDataset) (10% Sample):** A diverse collection of toxic comments, randomly downsampled to 10% to balance the distribution.

## Preprocessing & Cleaning
To ensure the data is "machine-ready" and consistent across different sources, the following transformations were applied:

Prefix Removal: Stripped leading bullet points (- ) from the beginning of every sentence to normalize text structure.

Escape Sequence Correction: Converted literal \\n- string sequences into standard newline characters and removed redundant leading dashes.

Column Pruning: Removed all auxiliary metadata columns, retaining only the core text and label columns (the last 3 columns of the original merged set).

Sampling & Merging: * Full train split of ToxiGen.

0.1 (10%) random sample of AiActivity/ToxicDataset.

## Shuffling: The final combined dataset was globally shuffled with a fixed seed (42) to ensure a diverse distribution of toxicity types across batches.

## Intended Use
Primary Use: Training and fine-tuning LLM safety guardrails.

Secondary Use: Benchmarking moderation APIs against historical and synthetic tropes.

## [!CAUTION]

Warning: This dataset contains highly offensive, biased, and dehumanizing language. It is intended strictly for research and development of safety tools. Use with professional discretion.

## Dataset Schema
The final dataset contains the following 3 columns:
| Column Name | Type | Description |
| :--- | :--- | :--- |
| input | string | The cleaned input |
| output | string | Output |
| label | int | Toxicity classification (0: Neutral, 1: Toxic). |