jordiclive/scored_summarization_datasets

# Dataset Card for "Scored-Summarization-datasets"
A collection of Text summarization datasets geared towards training a multi-purpose text summarizer.

Each dataset is a parquet file with the following features.

#### default
- `text`: a `string` feature. The `source` document
- `summary`: a `string` feature. The summary of the document
- `provenance`: a `string` feature. Information about the sub dataset.
- `t5_text_token_count`: a `int64` feature. The number of tokens the text is encoded in.
- `t5_summary_token_count `: a `int64` feature. The number of tokens the summary is encoded in.
- `contriever_cos`: a `float64` feature. The Cosine Similarity of the Contriever text embedding and Contriever summary embedding.

### Sub-datasets
- billsum
- cnn_dailymail/3.0.0
- multixscience
- newsroom
- samsum
- scitldr/AIC
- tldr-challenge
- wikihow
- xsum

Information about the Contriever model can be found here: https://github.com/facebookresearch/contriever.

原始信息汇总

数据集概述

数据集名称

Scored-Summarization-datasets

数据集目的

用于训练多用途文本摘要模型。

数据集结构

每个数据集以Parquet文件格式存储，包含以下特征：

默认特征

• text: 字符串类型，源文档内容。
• summary: 字符串类型，文档摘要。
• provenance: 字符串类型，子数据集信息。
• t5_text_token_count: 整数类型，文本编码的令牌数。
• t5_summary_token_count: 整数类型，摘要编码的令牌数。
• contriever_cos: 浮点数类型，Contriever文本嵌入与摘要嵌入的余弦相似度。

子数据集

• billsum
• cnn_dailymail/3.0.0
• multixscience
• newsroom
• samsum
• scitldr/AIC
• tldr-challenge
• wikihow
• xsum