anonymoustobi/OpenHermes-2.5

---
language:
- eng
pretty_name: OpenHermes 2.5
tags:
- synthetic
- GPT-4
- Distillation
- Compilation
---


# Dataset Card for Dataset Name

This is the dataset that made OpenHermes 2.5 and Nous Hermes 2 series of models.

Support me on GitHub sponsors <3 : https://github.com/sponsors/teknium1

## Dataset Details

### Dataset Description

The Open Hermes 2/2.5 and Nous Hermes 2 models have made significant advancements of SOTA LLM's over recent months, and are underpinned by this exact compilation and curation of many open source datasets and custom created synthetic datasets.

The Open Hermes 2.5 dataset is a continuation of the Open Hermes 1 dataset, at a much larger scale, much more diverse, and much higher quality compilation, reaching 1M, primarily synthetically generated instruction and chat samples.

## Lilac Integration

This dataset has been pushed to Lilac's (a data curation and exploration platform) live HuggingFace spaces, that hosts many popular OS Datasets for exploration and curation, as well as does Text Embedding searches and Clustering of those datasets

Check out that out here: https://lilacai-lilac.hf.space/datasets#lilac/OpenHermes-2.5

## Dataset Sources

### Airoboros 2.2
By Jon Durbin: https://huggingface.co/datasets/jondurbin/airoboros-2.2

### CamelAI Domain Expert Datasets (Physics, Math, Chemistry & Biology)
By CamelAI: https://huggingface.co/camel-ai

### ChatBot Arena (GPT-4 Only)
By LMSys: https://huggingface.co/datasets/lmsys/lmsys-chat-1m

### Collective Cognition (09-11-2023)
By Teknium: https://huggingface.co/datasets/CollectiveCognition/chats-data-2023-09-22

### CoT Alpaca GPT4
I have lost the source page for this dataset, sorry

### Evol Instruct 70K && 140K
By WizardLM:
70K: https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k
140k: https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k

### Glaive Code Assistant
By Sahil & Glaive: https://huggingface.co/datasets/glaiveai/glaive-code-assistant

### GPT4-LLM
By Baolin Peng*, Chunyuan Li*, Pengcheng He*, Michel Galley, Jianfeng Gao
https://huggingface.co/datasets/teknium/GPT4-LLM-Cleaned

### GPTeacher
By Teknium & Kuruminha: https://github.com/teknium1/GPTeacher

### Medical Tasks
By CogStack: https://github.com/CogStack/OpenGPT

### MetaMath 40k
By MetaMath: https://huggingface.co/datasets/meta-math/MetaMathQA

### SlimOrca 550K
By Wing Lian and Guan Wang and Bleys Goodson and Eugene Pentland and Austin Cook and Chanvichet Vong and Teknium:
https://huggingface.co/datasets/Open-Orca/SlimOrca

### Platypus
By Garage-bAInd: https://huggingface.co/datasets/garage-bAInd/Open-Platypus

### ShareGPT (GPT4-Only)
By LMSys: https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered

### Unnatural Instructions GPT4
By Baolin Peng*, Chunyuan Li*, Pengcheng He*, Michel Galley, Jianfeng Gao:
https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM

## Dataset Structure

The dataset follows a sharegpt structure. This means it is a list of dictionaries, with each dictionary containing a new list of dicts called conversations. Each turn in a conversation has two dictionaries, a "from" field, which denotes the role of that turn, and a "value" field which contains the actual text.

Here is an example of an entry:

```
{
"conversations": [
{
"from": "system",
"value": ""
},
{
"from": "human",
"value": "Every day, a tree drops 7 leaves. How many leaves would it drop in a month of February in a non-leap year? Include your logic."
},
{
"from": "gpt",
"value": "Here's the logic behind this:\n\n1. We know that February has 28 days in a non-leap year.\n2. If the tree drops 7 leaves every day, then over the course of February, it would drop:\n Leaves dropped in February = Leaves per day * Days in February\n = 7 leaves * 28 days\n = 196 leaves\n\nSo, the tree would drop 196 leaves in February in a non-leap year."
}
],
"source": "airoboros2.2",
"category": "orca"
},
```

Some examples, like the one above, contain the metadata that came with the dataset, such as "category" being "orca", and many contain the source dataset that it was curated from.

## Citation

```bibtex
@misc{OpenHermes 2.5,
title = {OpenHermes 2.5: An Open Dataset of Synthetic Data for Generalist LLM Assistants},
author = {Teknium},
year = {2023},
publisher = {HuggingFace},
url = {https://huggingface.co/datasets/teknium/OpenHermes-2.5}
}
```

language:
- 英语
pretty_name: OpenHermes 2.5
tags:
- 合成数据
- GPT-4
- 知识蒸馏
- 数据汇编
---


# 数据集卡片

本数据集为支撑OpenHermes 2.5与Nous Hermes 2系列模型研发的核心数据集。

欢迎通过GitHub赞助商渠道予以支持：https://github.com/sponsors/teknium1

## 数据集详情

### 数据集描述

近月来，Open Hermes 2/2.5与Nous Hermes 2系列模型在当前最优（SOTA）大语言模型（Large Language Model, LLM）领域取得了显著进展，其研发根基正是本数据集对大量开源数据集与定制合成数据集的精准汇编与精选整合。

OpenHermes 2.5数据集是OpenHermes 1数据集的延续，其汇编规模更大、样本多样性更丰富、质量更优异，总计包含约100万条以合成生成的指令与对话样本为主的数据集。

## Lilac 集成

本数据集已部署至Lilac（一款数据精选与探索平台）的HuggingFace在线空间，该空间汇聚了多款热门开源数据集，支持数据探索与精选操作，同时可对数据集进行文本嵌入搜索与聚类分析。

可通过以下链接访问该空间：https://lilacai-lilac.hf.space/datasets#lilac/OpenHermes-2.5

## 数据集来源

### Airoboros 2.2
作者：Jon Durbin，数据集地址：https://huggingface.co/datasets/jondurbin/airoboros-2.2

### CamelAI 领域专家数据集（物理、数学、化学与生物学）
作者：CamelAI，数据集地址：https://huggingface.co/camel-ai

### ChatBot Arena（仅GPT-4）
作者：LMSys，数据集地址：https://huggingface.co/datasets/lmsys/lmsys-chat-1m

### Collective Cognition（2023年9月11日）
作者：Teknium，数据集地址：https://huggingface.co/datasets/CollectiveCognition/chats-data-2023-09-22

### CoT Alpaca GPT4
本数据集的源页面已遗失，敬请谅解

### Evol Instruct 70K 与 140K 数据集
作者：WizardLM
70K版本地址：https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_70k
140K版本地址：https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k

### Glaive 代码助手数据集
作者：Sahil 与 Glaive，数据集地址：https://huggingface.co/datasets/glaiveai/glaive-code-assistant

### GPT4-LLM 数据集
作者：Baolin Peng*、Chunyuan Li*、Pengcheng He*、Michel Galley、Jianfeng Gao，数据集地址：https://huggingface.co/datasets/teknium/GPT4-LLM-Cleaned

### GPTeacher 数据集
作者：Teknium 与 Kuruminha，数据集地址：https://github.com/teknium1/GPTeacher

### 医疗任务数据集
作者：CogStack，数据集地址：https://github.com/CogStack/OpenGPT

### MetaMath 40K 数据集
作者：MetaMath，数据集地址：https://huggingface.co/datasets/meta-math/MetaMathQA

### SlimOrca 550K 数据集
作者：Wing Lian、Guan Wang、Bleys Goodson、Eugene Pentland、Austin Cook、Chanvichet Vong 与 Teknium，数据集地址：https://huggingface.co/datasets/Open-Orca/SlimOrca

### Platypus 数据集
作者：Garage-bAInd，数据集地址：https://huggingface.co/datasets/garage-bAInd/Open-Platypus

### ShareGPT（仅GPT-4）
作者：LMSys，数据集地址：https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered

### Unnatural Instructions GPT4 数据集
作者：Baolin Peng*、Chunyuan Li*、Pengcheng He*、Michel Galley、Jianfeng Gao，数据集地址：https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM

## 数据集结构

本数据集采用ShareGPT格式规范，即由一系列字典构成，每个字典包含一个名为`conversations`的子列表，该子列表存储对话轮次信息。每一轮对话包含两个字典字段：`from`字段用于标识该轮次的发言角色，`value`字段则存储具体的文本内容。

以下是一条数据条目的示例：

{
"conversations": [
{
"from": "system",
"value": ""
},
{
"from": "human",
"value": "Every day, a tree drops 7 leaves. How many leaves would it drop in a month of February in a non-leap year? Include your logic."
},
{
"from": "gpt",
"value": "Here's the logic behind this:

1. We know that February has 28 days in a non-leap year.
2. If the tree drops 7 leaves every day, then over the course of February, it would drop:
Leaves dropped in February = Leaves per day * Days in February
= 7 leaves * 28 days
= 196 leaves

So, the tree would drop 196 leaves in February in a non-leap year."
}
],
"source": "airoboros2.2",
"category": "orca"
},

部分示例条目包含数据集自带的元数据，例如上述示例中的`category`字段值为`orca`；多数条目还标注了其源自的精选数据集名称。

## 引用
bibtex
@misc{OpenHermes 2.5,
title = {OpenHermes 2.5: An Open Dataset of Synthetic Data for Generalist LLM Assistants},
author = {Teknium},
year = {2023},
publisher = {HuggingFace},
url = {https://huggingface.co/datasets/teknium/OpenHermes-2.5}
}