jigsaw-unintended-bias-in-toxicity

Background
At the end of 2017 the Civil Comments platform shut down and chose make their ~2m public comments from their platform available in a lasting open archive so that researchers could understand and improve civility in online conversations for years to come. Jigsaw sponsored this effort and extended annotation of this data by human raters for various toxic conversational attributes.

Labelling Schema
To obtain the toxicity labels, each comment was shown to up to 10 annotators*. Annotators were asked to: "Rate the toxicity of this comment"

Very Toxic (a very hateful, aggressive, or disrespectful comment that is very likely to make you leave a discussion or give up on sharing your perspective)
Toxic (a rude, disrespectful, or unreasonable comment that is somewhat likely to make you leave a discussion or give up on sharing your perspective)
Hard to Say
Not Toxic

For the current dataset from Kaggle https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/data
We have used the dataset all_data.csv which contains the following columns:
Index(['id', 'comment_text', 'split', 'created_date', 'publication_id',
'parent_id', 'article_id', 'rating', 'funny', 'wow', 'sad', 'likes',
'disagree', 'toxicity', 'severe_toxicity', 'obscene', 'sexual_explicit',
'identity_attack', 'insult', 'threat', 'male', 'female', 'transgender',
'other_gender', 'heterosexual', 'homosexual_gay_or_lesbian', 'bisexual',
'other_sexual_orientation', 'christian', 'jewish', 'muslim', 'hindu',
'buddhist', 'atheist', 'other_religion', 'black', 'white', 'asian',
'latino', 'other_race_or_ethnicity', 'physical_disability',
'intellectual_or_learning_disability', 'psychiatric_or_mental_illness',
'other_disability', 'identity_annotator_count',
'toxicity_annotator_count'])

Where the target column is 'toxicity' Values range from 0 to 1, where higher values indicate more toxic comments.
Other columns:
severe_toxicity, obscene, sexual_explicit, etc., are subcategories that provide more specific toxicity signals.
Demographic columns (e.g., male, black, christian) quantify mentions or perceived biases in comments.

paper_url = "https://papers.nips.cc/paper_files/paper/2022/file/9ca22870ae0ba55ee50ce3e2d269e5de-Paper-Datasets_and_Benchmarks.pdf"

original_data_url = "https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification"

We deleted the next columns: "['id', 'split', 'publication_id', 'parent_id', 'article_id']" because they are not needed for the analysis.