h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1

---
license: apache-2.0
language:
- en
thumbnail: https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources/images/favicon.ico
tags:
- gpt
- llm
- large language model
- open-source
---
# h2oGPT Data Card
## Summary

H2O.ai's `h2ogpt-oig-oasst1-instruct-cleaned-v1` is an open-source instruct-type dataset for fine-tuning of large language models, licensed for commercial use.

- Number of rows: `349837`
- Number of columns: `3`
- Column names: `['input', 'source', 'prompt_type']`

## Source

- [Original LAION OIG Dataset](https://github.com/LAION-AI/Open-Instruction-Generalist)
- [LAION OIG data detoxed and filtered down by scripts in h2oGPT repository](https://github.com/h2oai/h2ogpt/blob/main/FINETUNE.md#high-quality-oig-based-instruct-data)

- [Original Open Assistant data in tree structure](https://huggingface.co/datasets/OpenAssistant/oasst1)
- [This flattened dataset created by script in h2oGPT repository](https://github.com/h2oai/h2ogpt/blob/5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a/create_data.py#L1253)

原始信息汇总

h2oGPT Data Card 概述

基本信息

• 数据集名称: h2ogpt-oig-oasst1-instruct-cleaned-v1
• 许可证: Apache-2.0
• 语言: 英语
• 标签: GPT, LLM, 大型语言模型, 开源

数据集详情

• 行数: 349,837
• 列数: 3
• 列名: [input, source, prompt_type]

数据来源

• 原始数据集:
  • LAION OIG Dataset
  • Open Assistant data (oasst1)
• 数据处理:
  • 通过h2oGPT仓库中的脚本对LAION OIG数据进行净化和筛选。
  • 通过h2oGPT仓库中的脚本将Open Assistant数据集扁平化处理。