summedits

# Factual Consistency in Summarization

Can you tell which edits of summaries are consistent, and which are inconsistent?

<p align="center">
<img width="650" src="https://raw.githubusercontent.com/salesforce/factualNLG/master/images/summedits_examples.png">
</p>

## SummEdits Benchmark (Section 6-7)

We release the 6,348 samples of data for the 10 domains in the SummEdits. Each sample has entries for:
- `id`: a unique ID for the sample,
- `doc`: the input document,
- `summary`: the summary that is either consistent or inconsistent with the facts in the document,
- `label`: 1 if the summary is factually consistent, and 0 otherwise,
- `original_summary`: the (consistent) seed summary that was used as a starting point for the summary,
- `edit_types`: for summaries that are inconsistent, corresponds to GPT4 classified type of error.

For more detail on the data loading and benchmarking, we recommend you check out the Github repo: [https://github.com/salesforce/factualNLG](https://github.com/salesforce/factualNLG)

# 摘要生成中的事实一致性
你能否判别摘要的各类编辑版本是否符合事实一致性？

<p align="center">
<img width="650" src="https://raw.githubusercontent.com/salesforce/factualNLG/master/images/summedits_examples.png">
</p>

## SummEdits基准测试（第6-7节）
我们发布了SummEdits数据集覆盖10个领域的6348条数据样本。每条样本包含以下字段：
- `id`：该样本的唯一标识符
- `doc`：输入源文档
- `summary`：与源文档事实相符或相悖的生成摘要
- `label`：若该摘要符合事实一致性则取值为1，否则为0
- `original_summary`：用作该摘要生成起点的（符合事实的）种子摘要
- `edit_types`：针对存在事实不一致的摘要，对应由GPT-4分类的错误类型

如需了解数据加载与基准测试的更多细节，我们推荐您查阅该GitHub仓库：[https://github.com/salesforce/factualNLG](https://github.com/salesforce/factualNLG)