Isotonic/open-instruct-v1_deduped

---
dataset_info:
features:
- name: instruction
dtype: string
- name: input
dtype: string
- name: output
dtype: string
- name: text
dtype: string
splits:
- name: train
num_bytes: 398896451.6208479
num_examples: 229530
- name: test
num_bytes: 99724981.84707496
num_examples: 57383
download_size: 228255350
dataset_size: 498621433.4679228
task_categories:
- text-generation
- conversational
language:
- en
size_categories:
- 100K<n<1M
---
# Dataset Card for "open-instruct-v1_deduped"

- Deduplicated version of [Isotonic/open-instruct-v1](https://huggingface.co/datasets/Isotonic/open-instruct-v1)
- Deduplicated with min Jaccard similarity of 0.8
- Uses Stablility's System Prompt
```
### System: StableLM Tuned (Alpha version)
- StableLM is a helpful and harmless open-source AI language model developed by StabilityAI.
- StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes.
- StableLM will refuse to participate in anything that could harm a human.
```

[More Information needed](https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)

原始信息汇总

数据集概述

数据集名称

• open-instruct-v1_deduped

数据集特征

• instruction: 数据类型为字符串
• input: 数据类型为字符串
• output: 数据类型为字符串
• text: 数据类型为字符串

数据集分割

• train: 包含229530个样本，占用398896451.6208479字节
• test: 包含57383个样本，占用99724981.84707496字节

数据集大小

• 下载大小: 228255350字节
• 数据集总大小: 498621433.4679228字节

任务类别

• 文本生成
• 对话系统

语言

• 英语

大小类别

• 100K<n<1M