julep-ai/dfe-stacked_samsum

---
language:
- en
license: mit
task_categories:
- feature-extraction
pretty_name: Dialog-Fact Encod
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
- split: validation
path: data/validation-*
dataset_info:
features:
- name: dialogue
dtype: string
- name: summary
dtype: string
- name: is_truncated
dtype: bool
- name: is_augmented
dtype: bool
splits:
- name: train
num_bytes: 225951776.22338164
num_examples: 336975
- name: test
num_bytes: 25105976.423639305
num_examples: 37442
- name: validation
num_bytes: 27895380.35297907
num_examples: 41602
download_size: 174858508
dataset_size: 278953133.0
---
# Dataset Card for "dfe-stacked_samsum"

This custom dataset [julep-ai/dfe-stacked_samsum](https://huggingface.co/datasets/julep-ai/dfe-stacked_samsum) was created from [stacked-summaries/stacked-samsum-1024](https://huggingface.co/datasets/stacked-summaries/stacked-samsum-1024) by:
1. Extracting summaries for corresponding dialogs to emulate "facts"
2. Then truncating the dialogs to emulate "missing information"
3. And then augmenting the dialogs using LLMs to emulate "additional information"

It is used to train our [Dialog-Fact Encoder](https://huggingface.co/julep-ai/dfe-base-en) model.

> This dataset is permissively licensed under the MIT license.

## Notebooks

The data preparation process is documented in the [notebook](https://huggingface.co/datasets/julep-ai/dfe-stacked_samsum/blob/main/data_prep.ipynb) and you can also view the [rendered pdf](https://huggingface.co/datasets/julep-ai/dfe-stacked_samsum/blob/main/data_prep.pdf).

原始信息汇总

数据集概述

基本信息

• 语言: 英语
• 许可证: MIT
• 任务类别: 特征提取
• 数据集名称: Dialog-Fact Encod

配置信息

• 配置名称: default
  • 数据文件:
    • 训练集: data/train-*
    • 测试集: data/test-*
    • 验证集: data/validation-*

数据集信息

• 特征:

  • 对话: 字符串
  • 摘要: 字符串
  • 是否截断: 布尔值
  • 是否增强: 布尔值

• 拆分:

  • 训练集:
    • 字节数: 225951776.22338164
    • 样本数: 336975
  • 测试集:
    • 字节数: 25105976.423639305
    • 样本数: 37442
  • 验证集:
    • 字节数: 27895380.35297907
    • 样本数: 41602

• 下载大小: 174858508

• 数据集大小: 278953133.0