s-nlp/en_paradetox_toxicity

---
license: openrail++
task_categories:
- text-classification
language:
- en
---

# ParaDetox: Detoxification with Parallel Data (English). Toxicity Task Results

This repository contains information about **Toxicity Task** markup from [English Paradetox dataset](https://huggingface.co/datasets/s-nlp/paradetox) collection pipeline.
The original paper ["ParaDetox: Detoxification with Parallel Data"](https://aclanthology.org/2022.acl-long.469/) was presented at ACL 2022 main conference.

## ParaDetox Collection Pipeline

The ParaDetox Dataset collection was done via [Yandex.Toloka](https://toloka.yandex.com/) crowdsource platform. The collection was done in three steps:
* *Task 1:* **Generation of Paraphrases**: The first crowdsourcing task asks users to eliminate toxicity in a given sentence while keeping the content.
* *Task 2:* **Content Preservation Check**: We show users the generated paraphrases along with their original variants and ask them to indicate if they have close meanings.
* *Task 3:* **Toxicity Check**: Finally, we check if the workers succeeded in removing toxicity.

Specifically this repo contains the results of **Task 3: Toxicity Check**. Here, the samples with markup confidence >= 90 are present.
The input here is text and the label shows if the text is toxic or not.
Totally, datasets contains 26,507 samples. Among them, the minor part is toxic examples (4,009 pairs).

## Citation

```
@inproceedings{logacheva-etal-2022-paradetox,
title = "{P}ara{D}etox: Detoxification with Parallel Data",
author = "Logacheva, Varvara and
Dementieva, Daryna and
Ustyantsev, Sergey and
Moskovskiy, Daniil and
Dale, David and
Krotova, Irina and
Semenov, Nikita and
Panchenko, Alexander",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.acl-long.469",
pages = "6804--6818",
abstract = "We present a novel pipeline for the collection of parallel data for the detoxification task. We collect non-toxic paraphrases for over 10,000 English toxic sentences. We also show that this pipeline can be used to distill a large existing corpus of paraphrases to get toxic-neutral sentence pairs. We release two parallel corpora which can be used for the training of detoxification models. To the best of our knowledge, these are the first parallel datasets for this task.We describe our pipeline in detail to make it fast to set up for a new language or domain, thus contributing to faster and easier development of new parallel resources.We train several detoxification models on the collected data and compare them with several baselines and state-of-the-art unsupervised approaches. We conduct both automatic and manual evaluations. All models trained on parallel data outperform the state-of-the-art unsupervised models by a large margin. This suggests that our novel datasets can boost the performance of detoxification systems.",
}
```

## Contacts

For any questions, please contact: Daryna Dementieva ([e-mail])

许可证: openrail++
任务类别:
- 文本分类
语言:
- 英语

# ParaDetox：基于平行数据的文本去毒（英文）：毒性任务结果

本仓库收录了来自[英文ParaDetox数据集（English Paradetox dataset）](https://huggingface.co/datasets/s-nlp/paradetox) 标注流程中的**毒性任务**标注相关信息。原始论文《ParaDetox: Detoxification with Parallel Data》已在ACL 2022主会议上发表。

## ParaDetox数据集构建流程

该数据集通过[Yandex.Toloka](https://toloka.yandex.com/) 众包平台完成构建，共分为三个阶段：
* **任务1：释义生成**：首个众包任务要求参与者在保留原句核心语义的前提下，清除给定句子中的毒性表述。
* **任务2：语义一致性校验**：我们向标注人员展示生成的释义句及其原始毒性文本，要求其判断二者语义是否高度相近。
* **任务3：毒性校验**：最终环节用于验证标注人员是否成功移除了文本中的毒性内容。

具体而言，本仓库仅收录**任务3：毒性校验**的结果，其中仅保留标注置信度≥90%的样本。本次任务的输入为文本片段，标签用于标识该文本是否具有毒性。本数据集总计包含26507条样本，其中占比较少的为毒性样本（共4009对）。

## 引用

@inproceedings{logacheva-etal-2022-paradetox,
title = "{P}ara{D}etox: Detoxification with Parallel Data",
author = "Logacheva, Varvara and
Dementieva, Daryna and
Ustyantsev, Sergey and
Moskovskiy, Daniil and
Dale, David and
Krotova, Irina and
Semenov, Nikita and
Panchenko, Alexander",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.acl-long.469",
pages = "6804--6818",
abstract = "We present a novel pipeline for the collection of parallel data for the detoxification task. We collect non-toxic paraphrases for over 10,000 English toxic sentences. We also show that this pipeline can be used to distill a large existing corpus of paraphrases to get toxic-neutral sentence pairs. We release two parallel corpora which can be used for the training of detoxification models. To the best of our knowledge, these are the first parallel datasets for this task.We describe our pipeline in detail to make it fast to set up for a new language or domain, thus contributing to faster and easier development of new parallel resources.We train several detoxification models on the collected data and compare them with several baselines and state-of-the-art unsupervised approaches. We conduct both automatic and manual evaluations. All models trained on parallel data outperform the state-of-the-art unsupervised models by a large margin. This suggests that our novel datasets can boost the performance of detoxification systems.",
}

## 联系方式

如有任何疑问，请联系：达里娜·德门季耶娃（Daryna Dementieva），邮箱：[e-mail]

原始信息汇总

ParaDetox数据集概述

数据集基本信息

• 许可证: openrail++
• 任务类别: 文本分类
• 语言: 英语

数据集描述

• 名称: ParaDetox: Detoxification with Parallel Data (English)
• 任务: 毒性检测
• 数据收集方法: 通过Yandex.Toloka众包平台进行，分为三个步骤：
  • 任务1: 生成同义句，要求用户在不改变原意的情况下消除句子中的毒性。
  • 任务2: 内容保留检查，展示生成的同义句及其原始版本，询问用户两者是否意义相近。
  • 任务3: 毒性检查，检查工作者是否成功移除毒性。
• 数据集内容: 包含任务3: 毒性检查的结果，样本标记置信度大于等于90%。输入为文本，标签指示文本是否具有毒性。
• 数据集规模: 总计26,507个样本，其中毒性样本4,009对。

引用信息

@inproceedings{logacheva-etal-2022-paradetox, title = "{P}ara{D}etox: Detoxification with Parallel Data", author = "Logacheva, Varvara and Dementieva, Daryna and Ustyantsev, Sergey and Moskovskiy, Daniil and Dale, David and Krotova, Irina and Semenov, Nikita and Panchenko, Alexander", booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = may, year = "2022", address = "Dublin, Ireland", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2022.acl-long.469", pages = "6804--6818", abstract = "We present a novel pipeline for the collection of parallel data for the detoxification task. We collect non-toxic paraphrases for over 10,000 English toxic sentences. We also show that this pipeline can be used to distill a large existing corpus of paraphrases to get toxic-neutral sentence pairs. We release two parallel corpora which can be used for the training of detoxification models. To the best of our knowledge, these are the first parallel datasets for this task.We describe our pipeline in detail to make it fast to set up for a new language or domain, thus contributing to faster and easier development of new parallel resources.We train several detoxification models on the collected data and compare them with several baselines and state-of-the-art unsupervised approaches. We conduct both automatic and manual evaluations. All models trained on parallel data outperform the state-of-the-art unsupervised models by a large margin. This suggests that our novel datasets can boost the performance of detoxification systems.", }