SEACrowd/filipino_hatespeech_election

该数据集是Andrade等人爬取的1,696,613条推文的子集，这些推文发布于2015年11月至2016年5月期间，正值菲律宾总统选举的竞选期间。推文根据候选人姓名（如Binay、Duterte、Poe、Roxas和Santiago）和与选举相关的标签（如#Halalan2016、#Eleksyon2016和#PiliPinas2016）进行筛选。数据经过预处理，包括数据去标识化、URL移除、特殊字符处理、标准化、标签处理和分词，以便进行特征提取和分类。数据集支持的任务是辱骂性语言预测。

The dataset used in this study was a subset of the corpus 1,696,613 tweets crawled by Andrade et al. and posted from November 2015 to May 2016 during the campaign period for the Philippine presidential election. They were culled based on the presence of candidate names (e.g., Binay, Duterte, Poe, Roxas, and Santiago) and election-related hashtags (e.g.,#Halalan2016,#Eleksyon2016, and#PiliPinas2016). Data preprocessing was performed to prepare the tweets for feature extraction and classification. It consisted of the following steps: data de-identification, uniform resource locator (URL) removal, special character processing, normalization, hashtag processing, and tokenization. The dataset supports the task of abusive language prediction.

原始信息汇总

Filipino Hatespeech Election 数据集概述

基本信息

• 名称: Filipino Hatespeech Election
• 语言: 菲律宾语 (fil)
• 任务类别: 辱骂性语言预测 (Abusive Language Prediction)
• 标签: 辱骂性语言预测
• 数据集版本:
  • 源版本: 1.0.0
  • SEACrowd版本: 2024.06.20
• 许可证: 未知

数据来源

• 数据集描述: 该数据集是从André等人在2015年11月至2016年5月期间爬取的1,696,613条推文中提取的一个子集，这些推文发布于菲律宾总统选举的竞选期间。数据筛选基于候选人姓名（如Binay, Duterte, Poe, Roxas, Santiago）和选举相关标签（如#Halalan2016, #Eleksyon2016, #PiliPinas2016）。
• 数据预处理: 数据预处理包括去识别化、URL移除、特殊字符处理、归一化、标签处理和分词。

使用方法

使用 datasets 库

python from datasets import load_dataset dset = datasets.load_dataset("SEACrowd/filipino_hatespeech_election", trust_remote_code=True)

使用 seacrowd 库

python import seacrowd as sc

使用默认配置加载数据集

dset = sc.load_dataset("filipino_hatespeech_election", schema="seacrowd")

查看数据集的所有可用子集（配置名称）

print(sc.available_config_names("filipino_hatespeech_election"))

使用特定配置加载数据集

dset = sc.load_dataset_by_config_name(config_name="<config_name>")

引用

bibtex @article{Cabasag-2019-hate-speech, title={Hate speech in Philippine election-related tweets: Automatic detection and classification using natural language processing.}, author={Neil Vicente Cabasag, Vicente Raphael Chan, Sean Christian Lim, Mark Edward Gonzales, and Charibeth Cheng}, journal={Philippine Computing Journal}, volume={XIV}, number={1}, month={August}, year={2019} }

@article{lovenia2024seacrowd, title={SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages}, author={Holy Lovenia and Rahmad Mahendra and Salsabil Maulana Akbar and Lester James V. Miranda and Jennifer Santoso and Elyanah Aco and Akhdan Fadhilah and Jonibek Mansurov and Joseph Marvin Imperial and Onno P. Kampman and Joel Ruben Antony Moniz and Muhammad Ravi Shulthan Habibi and Frederikus Hudi and Railey Montalan and Ryan Ignatius and Joanito Agili Lopo and William Nixon and Börje F. Karlsson and James Jaya and Ryandito Diandaru and Yuze Gao and Patrick Amadeus and Bin Wang and Jan Christian Blaise Cruz and Chenxi Whitehouse and Ivan Halim Parmonangan and Maria Khelli and Wenyu Zhang and Lucky Susanto and Reynard Adha Ryanda and Sonny Lazuardi Hermawan and Dan John Velasco and Muhammad Dehan Al Kautsar and Willy Fitra Hendria and Yasmin Moslem and Noah Flynn and Muhammad Farid Adilazuarda and Haochen Li and Johanes Lee and R. Damanhuri and Shuo Sun and Muhammad Reza Qorib and Amirbek Djanibekov and Wei Qi Leong and Quyet V. Do and Niklas Muennighoff and Tanrada Pansuwan and Ilham Firdausi Putra and Yan Xu and Ngee Chia Tai and Ayu Purwarianti and Sebastian Ruder and William Tjhi and Peerat Limkonchotiwat and Alham Fikri Aji and Sedrick Keh and Genta Indra Winata and Ruochen Zhang and Fajri Koto and Zheng-Xin Yong and Samuel Cahyawijaya}, year={2024}, eprint={2406.10118}, journal={arXiv preprint arXiv: 2406.10118} }