有没有相关的论文或文献参考？
这个数据集是基于什么背景创建的？
数据集的作者是谁？
能帮我联系到这个数据集的作者吗？
这个数据集如何下载？
HATEDAY|仇恨言论检测数据集|多语言数据数据集
收藏arXiv2024-11-23 更新2024-11-27 收录
下载链接：
https://huggingface.co/datasets/manueltonneau/hateday
下载链接
链接失效反馈资源简介：
HATEDAY数据集是由牛津大学等机构创建的首个全球性仇恨言论数据集，旨在解决在线仇恨言论检测的全球挑战。该数据集包含240,000条标注的推文，随机抽样自2022年9月21日发布的所有推文，涵盖阿拉伯语、英语、法语、德语、印度尼西亚语、葡萄牙语、西班牙语和土耳其语八种语言，以及美国、印度、尼日利亚和肯尼亚四个国家。数据集的创建过程包括数据收集、过滤和标注，确保了语言和国家的代表性。HATEDAY数据集的应用领域主要集中在仇恨言论检测模型的评估和改进，旨在提高模型在实际社交媒体环境中的检测性能。
提供机构：
牛津大学, 世界银行, 纽约大学, Meedan, 麻省理工学院, 博科尼大学
创建时间：
2024-11-23
AI搜集汇总
数据集介绍
构建方式
HATEDAY数据集的构建基于2022年9月21日发布的所有推文，涵盖了八种语言和四个英语为主要语言的国家。数据集通过随机抽样方法，从这些推文中选取了240,000条推文，并进行了详细的标注。标注过程由36名标注员完成，每位标注员负责一种语言或国家，确保了数据集的多语言和多地域代表性。标注分为三类：仇恨性、冒犯性和中性，进一步细化了仇恨性推文的目标群体，以捕捉不同语言和国家的仇恨言论特征。
特点
HATEDAY数据集的显著特点在于其全球代表性和多语言覆盖，首次实现了对社交媒体环境中仇恨言论的全面捕捉。数据集不仅包含了八种主要语言的推文，还特别关注了四个英语国家的推文，揭示了仇恨言论在不同语言和文化背景下的表现差异。此外，数据集通过详细的标注和多轮审核，确保了标注的高质量和一致性，为后续的仇恨言论检测研究提供了坚实的基础。
使用方法
HATEDAY数据集适用于多种仇恨言论检测模型的评估和训练。研究者可以利用该数据集进行跨语言和跨国家的仇恨言论检测性能比较，探索模型在不同语言环境下的表现。此外，数据集还可用于开发和验证新的仇恨言论检测算法，特别是在处理多语言和多文化背景下的仇恨言论时。通过分析数据集中的标注信息，研究者可以深入理解仇恨言论的构成和分布，从而提出更有效的检测和干预策略。
背景与挑战
背景概述
HATEDAY数据集由牛津大学、世界银行、纽约大学等多机构联合创建，旨在应对全球在线仇恨言论的挑战。该数据集于2022年9月21日从Twitter上随机抽取了240,000条推文，涵盖八种语言和四个英语为主要语言的国家。HATEDAY的推出填补了现有评估数据集在地理和语言代表性上的空白，揭示了学术数据集在真实世界应用中的性能高估问题，尤其在非欧洲语言中表现尤为明显。这一研究不仅为跨语言和跨国家的仇恨言论检测提供了新的基准，还强调了未来检测模型在真实社交媒体环境中评估的必要性。
当前挑战
HATEDAY数据集面临的挑战主要包括两个方面。首先，学术数据集的系统性偏见导致仇恨言论检测模型在真实世界中的性能被高估，尤其是在非欧洲语言环境中。其次，构建过程中遇到的挑战包括语言和地理代表性的确保，以及模型在区分仇恨言论与攻击性言论上的困难。此外，学术研究中对某些仇恨言论目标的关注与真实世界中的实际分布不一致，进一步影响了模型的性能。这些挑战凸显了在真实社交媒体环境中评估和改进仇恨言论检测模型的迫切需求。
常用场景
经典使用场景
HATEDAY数据集的经典应用场景在于其为跨语言和跨国家的仇恨言论检测提供了宝贵的资源。通过分析2022年9月21日Twitter上发布的推文，该数据集涵盖了阿拉伯语、英语、法语、德语、印度尼西亚语、葡萄牙语、西班牙语和土耳其语等八种语言，以及美国、印度、尼日利亚和肯尼亚四个以英语为主要语言的国家。研究者利用这一数据集，可以深入探讨不同语言和文化背景下仇恨言论的流行程度和构成差异，从而开发和评估更为精准的仇恨言论检测模型。
实际应用
在实际应用中，HATEDAY数据集为社交媒体平台的内容审核提供了重要的参考。通过分析不同语言和国家的仇恨言论数据，平台可以更有效地调整其仇恨言论检测算法，以适应多样化的用户群体和语言环境。此外，该数据集还可以用于培训和测试人工智能模型，以提高其在多语言环境下的仇恨言论检测能力。最终，HATEDAY数据集的应用有助于提升社交媒体平台的内容审核效率和准确性，从而为用户提供一个更为安全和健康的在线环境。
衍生相关工作
HATEDAY数据集的发布催生了一系列相关研究工作，推动了仇恨言论检测领域的进一步发展。例如，研究者们利用该数据集开发了新的多语言仇恨言论检测模型，并探讨了如何在不同文化背景下优化这些模型。此外，HATEDAY还激发了对仇恨言论检测模型性能评估方法的重新思考，促使学术界和工业界更加关注模型在实际应用中的表现。未来，随着更多基于HATEDAY的研究成果的涌现，预计将会有更多创新的方法和技术被应用于仇恨言论检测领域。
以上内容由AI搜集并总结生成
数据主题
5,000+
优质数据集
54 个
任务类型
进入经典数据集
热门数据集
Google Scholar
Google Scholar是一个学术搜索引擎，旨在检索学术文献、论文、书籍、摘要和文章等。它涵盖了广泛的学科领域，包括自然科学、社会科学、艺术和人文学科。用户可以通过关键词搜索、作者姓名、出版物名称等方式查找相关学术资源。
scholar.google.com 收录
Population and Housing Census of 2007 - Ethiopia
Geographic coverage --------------------------- National coverage Analysis unit --------------------------- Household Person Housing unit Universe --------------------------- The census has counted people on dejure and defacto basis. The dejure population comprises all the persons who belong to a given area at a given time by virtue of usual residence, while under defacto approach people were counted as the residents of the place where they found. In the census, a person is said to be a usual resident of a household (and hence an area) if he/she has been residing in the household continuously for at least six months before the census day or intends to reside in the household for six months or longer. Thus, visitors are not included with the usual (dejure) population. Homeless persons were enumerated in the place where they spent the night on the enumeration day. The 2007 census counted foreign nationals who were residing in the city administration. On the other hand all Ethiopians living abroad were not counted. Kind of data --------------------------- Census/enumeration data [cen] Mode of data collection --------------------------- Face-to-face [f2f] Research instrument --------------------------- Two type sof questionnaires were used to collect census data: i) Short questionnaire ii) Long questionnaire Unlike the previous censuses, the contents of the short and long questionnaires were similar both for the urban and rural areas as well as for the entire city. But the short and the long questionnaires differ by the number of variables they contained. That is, the short questionnaire was used to collect basic data on population characteristics, such as population size, sex, age, language, ethnic group, religion, orphanhood and disability. Whereas the long questionnaire includes information on marital status, education, economic activity, migration, fertility, mortality, as well as housing stocks and conditions in addition to those questions contained in a short questionnaire.
catalog.ihsn.org 收录
CatMeows
该数据集包含440个声音样本，由21只属于两个品种（缅因州库恩猫和欧洲短毛猫）的猫在三种不同情境下发出的喵声组成。这些情境包括刷毛、在陌生环境中隔离和等待食物。每个声音文件都遵循特定的命名约定，包含猫的唯一ID、品种、性别、猫主人的唯一ID、录音场次和发声计数。此外，还有一个额外的zip文件，包含被排除的录音（非喵声）和未剪辑的连续发声序列。
huggingface 收录
Canadian Census
**Overview** The data package provides demographics for Canadian population groups according to multiple location categories: Forward Sortation Areas (FSAs), Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs), Federal Electoral Districts (FEDs), Health Regions (HRs) and provinces. **Description** The data are available through the Canadian Census and the National Household Survey (NHS), separated or combined. The main demographic indicators provided for the population groups, stratified not only by location but also for the majority by demographical and socioeconomic characteristics, are population number, females and males, usual residents and private dwellings. The primary use of the data at the Health Region level is for health surveillance and population health research. Federal and provincial departments of health and human resources, social service agencies, and other types of government agencies use the information to monitor, plan, implement and evaluate programs to improve the health of Canadians and the efficiency of health services. Researchers from various fields use the information to conduct research to improve health. Non-profit health organizations and the media use the health region data to raise awareness about health, an issue of concern to all Canadians. The Census population counts for a particular geographic area representing the number of Canadians whose usual place of residence is in that area, regardless of where they happened to be on Census Day. Also included are any Canadians who were staying in that area on Census Day and who had no usual place of residence elsewhere in Canada, as well as those considered to be 'non-permanent residents'. National Household Survey (NHS) provides demographic data for various levels of geography, including provinces and territories, census metropolitan areas/census agglomerations, census divisions, census subdivisions, census tracts, federal electoral districts and health regions. In order to provide a comprehensive overview of an area, this product presents data from both the NHS and the Census. NHS data topics include immigration and ethnocultural diversity; aboriginal peoples; education and labor; mobility and migration; language of work; income and housing. 2011 Census data topics include population and dwelling counts; age and sex; families, households and marital status; structural type of dwelling and collectives; and language. The data are collected for private dwellings occupied by usual residents. A private dwelling is a dwelling in which a person or a group of persons permanently reside. Information for the National Household Survey does not include information for collective dwellings. Collective dwellings are dwellings used for commercial, institutional or communal purposes, such as a hotel, a hospital or a work camp. **Benefits** - Useful for canada public health stakeholders, for public health specialist or specialized public and other interested parties. for health surveillance and population health research. for monitoring, planning, implementation and evaluation of health-related programs. media agencies may use the health regions data to raise awareness about health, an issue of concern to all canadians. giving the addition of longitude and latitude in some of the datasets the data can be useful to transpose the values into geographical representations. the fields descriptions along with the dataset description are useful for the user to quickly understand the data and the dataset. **License Information** The use of John Snow Labs datasets is free for personal and research purposes. For commercial use please subscribe to the [Data Library](https://www.johnsnowlabs.com/marketplace/) on John Snow Labs website. The subscription will allow you to use all John Snow Labs datasets and data packages for commercial purposes. **Included Datasets** - [Canadian Population and Dwelling by FSA 2011](https://www.johnsnowlabs.com/marketplace/canadian-population-and-dwelling-by-fsa-2011) - This Canadian Census dataset covers data on population, total private dwellings and private dwellings occupied by usual residents by forward sortation area (FSA). It is enriched with the percentage of the population or dwellings versus the total amount as well as the geographical area, province, and latitude and longitude. The whole Canada's population is marked as 100, referring to 100% for the percentages. - [Detailed Canadian Population Statistics by CMAs and CAs 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-cmas-and-cas-2011) - This dataset covers the population statistics of Canada by Census Metropolitan Areas (CMAs) and Census Agglomerations (CAs). It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by FED 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-fed-2011) - This dataset covers the population statistics of Canada from 2011 by Federal Electoral District of 2013 Representation Order. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Health Region 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-health-region-2011) - This dataset covers the population statistics of Canada by health region. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. - [Detailed Canadian Population Statistics by Province 2011](https://www.johnsnowlabs.com/marketplace/detailed-canadian-population-statistics-by-province-2011) - This dataset covers the population statistics of Canada by provinces and territories. It is categorized also by citizen/immigration status, ethnic origin, religion, mobility, education, language, work, housing, income etc. There is detailed characteristics categorization within these stated categories that are in 5 layers. **Data Engineering Overview** **We deliver high-quality data** - Each dataset goes through 3 levels of quality review - 2 Manual reviews are done by domain experts - Then, an automated set of 60+ validations enforces every datum matches metadata & defined constraints - Data is normalized into one unified type system - All dates, unites, codes, currencies look the same - All null values are normalized to the same value - All dataset and field names are SQL and Hive compliant - Data and Metadata - Data is available in both CSV and Apache Parquet format, optimized for high read performance on distributed Hadoop, Spark & MPP clusters - Metadata is provided in the open Frictionless Data standard, and its every field is normalized & validated - Data Updates - Data updates support replace-on-update: outdated foreign keys are deprecated, not deleted **Our data is curated and enriched by domain experts** Each dataset is manually curated by our team of doctors, pharmacists, public health & medical billing experts: - Field names, descriptions, and normalized values are chosen by people who actually understand their meaning - Healthcare & life science experts add categories, search keywords, descriptions and more to each dataset - Both manual and automated data enrichment supported for clinical codes, providers, drugs, and geo-locations - The data is always kept up to date – even when the source requires manual effort to get updates - Support for data subscribers is provided directly by the domain experts who curated the data sets - Every data source’s license is manually verified to allow for royalty-free commercial use and redistribution. **Need Help?** If you have questions about our products, contact us at [info@johnsnowlabs.com](mailto:info@johnsnowlabs.com).
Databricks 收录
MultiTalk
MultiTalk数据集是由韩国科学技术院创建，包含超过420小时的2D视频，涵盖20种不同语言，旨在解决多语言环境下3D说话头生成的问题。该数据集通过自动化管道从YouTube收集，每段视频都配有语言标签和伪转录，部分视频还包含伪3D网格顶点。数据集的创建过程包括视频收集、主动说话者验证和正面人脸验证，确保数据质量。MultiTalk数据集的应用领域主要集中在提升多语言3D说话头生成的准确性和表现力，通过引入语言特定风格嵌入，使模型能够捕捉每种语言独特的嘴部运动。
arXiv 收录