eduagarcia/FactNews

---
dataset_info:
- config_name: bias_prediction
features:
- name: file
dtype: string
- name: id_sente
dtype: string
- name: id_article
dtype: string
- name: domain
dtype: string
- name: year
dtype: string
- name: sentences
dtype: string
- name: label
dtype: int64
- name: label_text
dtype: string
splits:
- name: train
num_bytes: 163041
num_examples: 738
- name: full_train
num_bytes: 951010
num_examples: 4403
- name: test
num_bytes: 384327
num_examples: 1788
download_size: 718605
dataset_size: 1498378
- config_name: factuality_prediction
features:
- name: file
dtype: string
- name: id_sente
dtype: string
- name: id_article
dtype: string
- name: domain
dtype: string
- name: year
dtype: string
- name: sentences
dtype: string
- name: label
dtype: int64
- name: label_text
dtype: string
splits:
- name: train
num_bytes: 606722
num_examples: 2826
- name: full_train
num_bytes: 944929
num_examples: 4403
- name: test
num_bytes: 381863
num_examples: 1788
download_size: 927856
dataset_size: 1933514
- config_name: original
features:
- name: file
dtype: string
- name: id_sente
dtype: string
- name: id_article
dtype: string
- name: domain
dtype: string
- name: year
dtype: string
- name: sentences
dtype: string
- name: classe
dtype: int64
- name: label_text
dtype: string
splits:
- name: train
num_bytes: 1317047
num_examples: 6191
download_size: 516651
dataset_size: 1317047
configs:
- config_name: bias_prediction
data_files:
- split: train
path: bias_prediction/train-*
- split: full_train
path: bias_prediction/full_train-*
- split: test
path: bias_prediction/test-*
- config_name: factuality_prediction
data_files:
- split: train
path: factuality_prediction/train-*
- split: full_train
path: factuality_prediction/full_train-*
- split: test
path: factuality_prediction/test-*
- config_name: original
data_files:
- split: train
path: original/train-*
license: unknown
task_categories:
- text-classification
language:
- pt
- por
pretty_name: FactNews
size_categories:
- 1K<n<10K
multilinguality:
- monolingual
language_creators:
- found
annotations_creators:
- expert-generated
tags:
- subjectivity
- mediabias
- media-bias
---

## Disclaimer

*I am not the author of this dataset, this is a modified version of the FactCheck dataset on HuggingFace, the original data is made avaliable by Vargas et. al, 2023 and can be downloaded from the link: https://github.com/franciellevargas/FactNews*

*Modifications:*
- *The "original" subset contains the unmodified original CSV*
- *The subsets for the task of "bias_prediction" and "factuality_prediction" were splited between train (70%) AND test (30%) by randomly selecting
sentences grouped by their id_article. This configuration difers from the authors, who made a 90%/10% 10-fold split on the papers.*
- *Each task contains an unbalanced split (full-train) and the balanced-split (train)*

# Sentence-Level Annotated Dataset for Predicting Factuality of News and Bias of Media Outlets in Portuguese

Automated fact-checking and news credibility verification at scale require accurate prediction of news factuality and media bias.
Here, we introduce a large sentence-level dataset, titled FactNews, composed of 6,191 sentences expertly annotated according to factuality
and media bias definitions proposed by AllSides. We used the FactNews to assess the overall reliability of news sources by formulating two
text classification problems for predicting sentence-level factuality of news reporting and bias of media outlets. Our experiments demonstrate
that biased sentences present a higher number of words compared to factual sentences, besides having a predominance of emotions. Hence,
the fine-grained analysis of subjectivity and impartiality of news articles showed promising results for predicting the reliability of the
entire media outlet. Finally, due to the severity of fake news and political polarization in Brazil, and the lack of research for Portuguese,
both dataset and baseline were proposed for Brazilian Portuguese. The following table describes in detail the FactNews labels, documents, and stories:

| Factual| Quotes | Biased | Total sentences | Total news stories | Total news documents |
| :--- | :---: | ---: | ---: | ---: | ---: |
| 4,242 | 1,391 | 558 | 6,161 | 100 | 300 |

### Sources:
- Media 1: Folha de São Paulo
- Media 2: Estadão
- Media 3: O Globo

### Paper Results:

Sentence-Level Media Bias Prediction (90%/10% 10-fold split)
- 67% (F1-Score) by Fine-tuned mBert-case

Sentence-Level Factuality Prediction (90%/10% 10-fold split)
- 88% (F1-Score) by Fine-tuned mBert-case

## Citation

```
Vargas, F., Jaidka, K., Pardo, T.A.S., Benevenuto, F. (2023). Predicting Sentence-Level Factuality of News and Bias of Media Outlets. Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pp.1197--1206. Varna, Bulgaria. Association for Computational Linguistics (ACL).

```

**Bibtex**
```
@inproceedings{vargas-etal-2023-predicting,
title = "Predicting Sentence-Level Factuality of News and Bias of Media Outlets",
author = "Vargas, Francielle and
Jaidka, Kokil and
Pardo, Thiago and
Benevenuto, Fabr{\'\i}cio",
editor = "Mitkov, Ruslan and
Angelova, Galia",
booktitle = "Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing",
month = sep,
year = "2023",
address = "Varna, Bulgaria",
publisher = "INCOMA Ltd., Shoumen, Bulgaria",
url = "https://aclanthology.org/2023.ranlp-1.127",
pages = "1197--1206",
}
```
## Dataset Description

- **Homepage:** https://github.com/franciellevargas/FactNews
- **Paper:** [Predicting Sentence-Level Factuality of News and Bias of Media Outlets](https://aclanthology.org/2023.ranlp-1.127)

原始信息汇总

数据集概述

数据集名称

• 名称: FactNews
• 语言: 葡萄牙语（pt, por）
• 任务类别: 文本分类
• 多语言性: 单语种
• 数据集大小: 1K<n<10K

数据集配置

1. bias_prediction

   • 特征:
     • file: 字符串
     • id_sente: 字符串
     • id_article: 字符串
     • domain: 字符串
     • year: 字符串
     • sentences: 字符串
     • label: int64
     • label_text: 字符串
   • 分割:
     • train: 738个样本, 163041字节
     • full_train: 4403个样本, 951010字节
     • test: 1788个样本, 384327字节
   • 下载大小: 718605字节
   • 数据集大小: 1498378字节

2. factuality_prediction

   • 特征:
     • file: 字符串
     • id_sente: 字符串
     • id_article: 字符串
     • domain: 字符串
     • year: 字符串
     • sentences: 字符串
     • label: int64
     • label_text: 字符串
   • 分割:
     • train: 2826个样本, 606722字节
     • full_train: 4403个样本, 944929字节
     • test: 1788个样本, 381863字节
   • 下载大小: 927856字节
   • 数据集大小: 1933514字节

3. original

   • 特征:
     • file: 字符串
     • id_sente: 字符串
     • id_article: 字符串
     • domain: 字符串
     • year: 字符串
     • sentences: 字符串
     • classe: int64
     • label_text: 字符串
   • 分割:
     • train: 6191个样本, 1317047字节
   • 下载大小: 516651字节
   • 数据集大小: 1317047字节

数据集文件

• bias_prediction:

  • train: bias_prediction/train-*
  • full_train: bias_prediction/full_train-*
  • test: bias_prediction/test-*

• factuality_prediction:

  • train: factuality_prediction/train-*
  • full_train: factuality_prediction/full_train-*
  • test: factuality_prediction/test-*

• original:

  • train: original/train-*

数据集标签统计

• 总句子数: 6,161
• 总新闻故事数: 100
• 总新闻文档数: 300
• 事实句子数: 4,242
• 引用句子数: 1,391
• 偏见句子数: 558

来源媒体

• Media 1: Folha de São Paulo
• Media 2: Estadão
• Media 3: O Globo

论文结果

• 句子级媒体偏见预测: 67% F1-Score（Fine-tuned mBert-case）
• 句子级事实预测: 88% F1-Score（Fine-tuned mBert-case）

搜集汇总

数据集介绍

构建方式

FactNews数据集由Vargas等人于2023年构建，旨在支持葡萄牙语新闻事实性与媒体偏见预测的研究。该数据集以AllSides定义的事实性与媒体偏见标准为指导，从Folha de São Paulo、Estadão和O Globo三家巴西主流媒体中，精心挑选了100篇新闻报道，共计300篇新闻文档，并从中提取了6,191个句子。每个句子均由领域专家依据事实性、引用性和偏见性三类标签进行人工标注。在原始数据基础上，本版本进一步划分出bias_prediction和factuality_prediction两个子集，分别对应偏见预测与事实性预测任务，每个子集均按文章ID随机划分为70%训练集和30%测试集，并额外提供了不平衡的full_train与平衡的train分割。

特点

FactNews数据集的核心特点在于其细粒度的句子级标注，能够支持对新闻事实性与媒体偏见的精准建模。标注体系涵盖事实性、引用性和偏见性三类，其中事实性句子占比最高（4,242句），引用句次之（1,391句），偏见句最少（558句），这种分布反映了真实新闻语料中客观陈述占主导的语言特征。数据集还提供了丰富的元信息，包括句子所属的新闻文档ID、文章ID、来源媒体域名及年份，便于进行跨领域和时序分析。实验表明，偏见句通常比事实句包含更多词汇且情感色彩更浓，这为利用主观性分析评估媒体可靠性提供了有力依据。

使用方法

FactNews数据集适用于文本分类任务的训练与评估，尤其针对葡萄牙语新闻的事实性与媒体偏见预测。用户可通过HuggingFace Datasets库加载三个配置：original（原始完整数据）、bias_prediction（偏见预测任务）和factuality_prediction（事实性预测任务）。每个配置均提供train、full_train和test分割，其中bias_prediction和factuality_prediction的train为平衡子集，full_train为不平衡子集，用户可根据研究需求选择使用。数据集以CSV格式存储，包含句子文本、标签及元数据字段，可直接用于微调多语言BERT等预训练模型，或作为基准测试的评估数据。

背景与挑战

背景概述

FactNews数据集由Vargas等研究人员于2023年创建，隶属于巴西圣保罗大学和新加坡南洋理工大学等机构，旨在解决葡萄牙语新闻真实性评估与媒体偏见预测的核心研究问题。该数据集基于AllSides提出的真实性与媒体偏见定义，精心标注了6,191个句子，涵盖事实性、引语和偏见三类标签，数据源自巴西三大主流媒体（Folha de São Paulo、Estadão和O Globo）。FactNews的提出填补了葡萄牙语领域在细粒度新闻可靠性分析上的空白，为自动化事实核查和媒体可信度评估提供了关键基准，其句子级标注方式显著提升了模型对新闻主观性与公正性的捕捉能力，对巴西乃至葡语国家的假新闻治理与政治极化研究产生了深远影响。

当前挑战

FactNews数据集面临的挑战主要体现在两个方面。在领域问题层面，它致力于解决新闻事实性与媒体偏见的细粒度预测，但偏见句子通常包含更多词汇和情感色彩，导致模型在区分事实与偏见时易受语言歧义和语境复杂性的干扰，现有基线方法（如mBert微调）在偏见预测上的F1分数仅为67%，远低于事实性预测的88%，凸显了领域内对鲁棒性算法的迫切需求。在构建过程中，数据集虽采用专家标注以确保标签质量，但原始数据分布不均衡（事实性句子占多数，偏见句子仅558条），且句子按文章分组随机分割时需平衡训练与测试集的代表性，这增加了跨域泛化的难度，同时葡萄牙语资源的稀缺性进一步限制了模型性能的提升空间。

常用场景

经典使用场景

在自然语言处理与计算新闻学交叉领域，FactNews数据集以其精细的句子级标注体系，为新闻事实性与媒体偏见研究提供了坚实的基准资源。该数据集源自巴西三大主流媒体，包含6191条经专家依据AllSides标准标注的句子，分别对应事实性、引用性与偏向性三类标签。其经典使用场景聚焦于两项文本分类任务：一是预测新闻报道的事实性程度，二是识别媒体机构的立场偏见。通过构建基于mBERT的微调模型，研究者能够在句子粒度上量化新闻的主观性与客观性，从而为自动化事实核查系统的开发奠定基础。

实际应用

在实际应用中，FactNews数据集为新闻聚合平台、社交媒体内容审核系统以及媒体监测工具提供了关键支撑。基于该数据集训练的模型能够自动识别新闻报道中的偏见表达与事实偏离，辅助编辑团队进行内容质量把控。在巴西，该资源已被用于评估主流媒体的报道平衡性，并为公共舆论研究提供量化依据。此外，它还可集成至浏览器插件或移动应用中，实时向用户提示新闻片段的潜在偏见，提升公众的媒介素养与信息甄别能力。

衍生相关工作

FactNews数据集衍生了一系列具有影响力的学术工作。Vargas等人（2023）基于该数据集提出了句子级事实性与偏见预测的基线模型，在10折交叉验证中分别达到88%与67%的F1分数。后续研究进一步探索了多任务学习框架，将事实性与偏见预测联合优化，提升了整体性能。另有工作利用该数据集验证了情感分析特征在偏见检测中的有效性，并对比了不同预训练语言模型（如BERTimbau）在葡萄牙语新闻分析上的表现。这些衍生工作共同丰富了计算新闻学的方法论工具箱。