jigsaw-toxic-comment-classification-challenge

Discussing things you care about can be difficult. The threat of abuse and harassment online means that many people stop expressing themselves and give up on seeking different opinions. Platforms struggle to effectively facilitate conversations, leading many communities to limit or completely shut down user comments.

The Conversation AI team, a research initiative founded by Jigsaw and Google (both a part of Alphabet) are working on tools to help improve online conversation. One area of focus is the study of negative online behaviors, like toxic comments (i.e. comments that are rude, disrespectful or otherwise likely to make someone leave a discussion). So far they've built a range of publicly available models served through the Perspective API, including toxicity. But the current models still make errors, and they don't allow users to select which types of toxicity they're interested in finding (e.g. some platforms may be fine with profanity, but not with other types of toxic content).

This dataset provided a large number of Wikipedia comments which have been labeled by human raters for toxic behavior. The types of toxicity are:

toxic
severe_toxic
obscene
threat
insult
identity_hate

This is originally a multi-label classification problem. However, for the purpose of this task, we have combined the labels into a single column.

The dataset is split into a training set and a test set. If you want stick to multi-label classification, please refer to the original dataset on Kaggle https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/overview.
This is what we did to combine the labels into a single column:

Filter Valid Test Labels: In the test_labels_df, comments with labels of -1 are invalid and should be removed. These rows don't have meaningful toxicity annotations.

Create a Multiclass Target Column: Encode the six toxicity labels (toxic, severe_toxic, obscene, threat, insult, identity_hate) as a single class.

# Combine labels into a single column
def encode_multiclass(row):
return ','.join(row.index[row == 1])

# Apply encoding to both train and test datasets
for df in [train_df, test_df]:
df['class'] = df[['toxic', 'severe_toxic', 'obscene', 'threat', 'insult', 'identity_hate']].apply(encode_multiclass, axis=1)

Here, each combination of the six labels results in a unique class. For example, a comment with the labels toxic and obscene will have a class of toxic,obscene.
And so on.

Drop Original Labels after creating the class column.

paper_url = "https://papers.nips.cc/paper_files/paper/2022/file/9ca22870ae0ba55ee50ce3e2d269e5de-Paper-Datasets_and_Benchmarks.pdf"

original_data_url = "https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/overview"