WizardLM_evol_instruct_V2_196k

## News

- 🔥 🔥 🔥 [08/11/2023] We release **WizardMath** Models.
- 🔥 Our **WizardMath-70B-V1.0** model slightly outperforms some closed-source LLMs on the GSM8K, including **ChatGPT 3.5**, **Claude Instant 1** and **PaLM 2 540B**.
- 🔥 Our **WizardMath-70B-V1.0** model achieves **81.6 pass@1** on the [GSM8k Benchmarks](https://github.com/openai/grade-school-math), which is **24.8** points higher than the SOTA open-source LLM.
- 🔥 Our **WizardMath-70B-V1.0** model achieves **22.7 pass@1** on the [MATH Benchmarks](https://github.com/hendrycks/math), which is **9.2** points higher than the SOTA open-source LLM.

| Model | Checkpoint | Paper | GSM8k | MATH |Online Demo| License|
| ----- |------| ---- |------|-------| ----- | ----- |
| WizardMath-70B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-70B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a>| **81.6** | **22.7** |[Demo](http://47.103.63.15:50083/)| <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 </a> |
| WizardMath-13B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-13B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a>| **63.9** | **14.0** |[Demo](http://47.103.63.15:50082/)| <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 </a> |
| WizardMath-7B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-7B-V1.0" target="_blank">HF Link</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a>| **54.9** | **10.7** | [Demo](http://47.103.63.15:50080/)| <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 </a>|

<font size=4>

| ^Model | ^Checkpoint | ^Paper |^MT-Bench | ^AlpacaEval | ^WizardEval | ^HumanEval | ^License|
| ----- |------| ---- |------|-------| ----- | ----- | ----- |
| ^{WizardLM-13B-V1.2} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.2" target="_blank">HF Link</a>}| | ^7.06 | ^89.17% | ^101.4%|^{36.6 pass@1}|^{<a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 License </a>} |
| ^{WizardLM-13B-V1.1} |^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.1" target="_blank">HF Link</a>} | | ^6.76 |^86.32% | ^99.3% |^{25.0 pass@1}| ^{Non-commercial}|
| ^{WizardLM-30B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-30B-V1.0" target="_blank">HF Link</a>} | | ^7.01 | | ^97.8% | ^{37.8 pass@1}| ^{Non-commercial} |
| ^{WizardLM-13B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.0" target="_blank">HF Link</a>} | | ^6.35 | ^75.31% | ^89.1% |^{24.0 pass@1} | ^{Non-commercial}|
| ^{WizardLM-7B-V1.0}| ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-7B-V1.0" target="_blank">HF Link</a>} |^{📃 <a href="https://arxiv.org/abs/2304.12244" target="_blank">[WizardLM]</a>}| | | ^78.0% |^{19.1 pass@1}|^{Non-commercial}|
| ^{WizardCoder-15B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-15B-V1.0" target="_blank">HF Link</a>} | ^{📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a>} | || |^{57.3 pass@1} | ^{<a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement" target="_blank">OpenRAIL-M</a>} |
</font>

## 示例代码
```python
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load('AI-ModelScope/WizardLM_evol_instruct_V2_196k',subset_name='default', split='train', download_mode=DownloadMode.FORCE_REDOWNLOAD)
print(next(iter(ds)))
```

**Repository**: https://github.com/nlpxucan/WizardLM

**Twitter**: https://twitter.com/WizardLM_AI/status/1669364947606982656

This datasets contains 143K mixture evolved data of Alpaca and ShareGPT.

This is the latest optimized version of Evol-Instruct training data of WizardLM model.

Due to the data usage license, please **merge** the original [ShareGPT](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered) with this one to get the **final full-dataset**, which would consist of around 196k rows of data.

## 动态
- 🔥 🔥 🔥 [2023/08/11] 我们正式发布**WizardMath**系列模型。
- 🔥 我们的**WizardMath-70B-V1.0**模型在GSM8K基准测试中性能略优于部分闭源大语言模型（Large Language Model，LLM），包括**ChatGPT 3.5**、**Claude Instant 1**以及**PaLM 2 540B**。
- 🔥 我们的**WizardMath-70B-V1.0**模型在[GSM8K基准测试](https://github.com/openai/grade-school-math)上的`pass@1`指标达到**81.6**，较当前开源大语言模型的最优性能高出**24.8**个百分点。
- 🔥 我们的**WizardMath-70B-V1.0**模型在[MATH基准测试](https://github.com/hendrycks/math)上的`pass@1`指标达到**22.7**，较当前开源大语言模型的最优性能高出**9.2**个百分点。

| 模型 | 模型权重 | 论文 | GSM8K | MATH | 在线演示 | 授权协议 |
| ----- | ------ | ---- | ------ | ------- | ----- | ----- |
| WizardMath-70B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-70B-V1.0" target="_blank">Hugging Face 链接</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a> | **81.6** | **22.7** | [演示](http://47.103.63.15:50083/) | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 授权协议</a> |
| WizardMath-13B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-13B-V1.0" target="_blank">Hugging Face 链接</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a> | **63.9** | **14.0** | [演示](http://47.103.63.15:50082/) | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 授权协议</a> |
| WizardMath-7B-V1.0 | 🤗 <a href="https://huggingface.co/WizardLM/WizardMath-7B-V1.0" target="_blank">Hugging Face 链接</a> | 📃 <a href="https://arxiv.org/abs/2308.09583" target="_blank">[WizardMath]</a> | **54.9** | **10.7** | [演示](http://47.103.63.15:50080/) | <a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 授权协议</a> |

<font size=4>

| ^模型 | ^模型权重 | ^论文 |^MT-Bench | ^AlpacaEval | ^WizardEval | ^HumanEval | ^授权协议|
| ----- |------| ---- |------|-------| ----- | ----- | ----- |
| ^{WizardLM-13B-V1.2} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.2" target="_blank">Hugging Face 链接</a>}| | ^7.06 | ^89.17% | ^101.4%|^{36.6 pass@1}|^{<a href="https://ai.meta.com/resources/models-and-libraries/llama-downloads/" target="_blank">Llama 2 授权协议</a>} |
| ^{WizardLM-13B-V1.1} |^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.1" target="_blank">Hugging Face 链接</a>} | | ^6.76 |^86.32% | ^99.3% |^{25.0 pass@1}| ^非商用|
| ^{WizardLM-30B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-30B-V1.0" target="_blank">Hugging Face 链接</a>} | | ^7.01 | | ^97.8% | ^{37.8 pass@1}| ^非商用 |
| ^{WizardLM-13B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-13B-V1.0" target="_blank">Hugging Face 链接</a>} | | ^6.35 | ^75.31% | ^89.1% |^{24.0 pass@1} | ^非商用|
| ^{WizardLM-7B-V1.0}| ^{🤗 <a href="https://huggingface.co/WizardLM/WizardLM-7B-V1.0" target="_blank">Hugging Face 链接</a>} |^{📃 <a href="https://arxiv.org/abs/2304.12244" target="_blank">[WizardLM]</a>}| | | ^78.0% |^{19.1 pass@1}|^非商用|
| ^{WizardCoder-15B-V1.0} | ^{🤗 <a href="https://huggingface.co/WizardLM/WizardCoder-15B-V1.0" target="_blank">Hugging Face 链接</a>} | ^{📃 <a href="https://arxiv.org/abs/2306.08568" target="_blank">[WizardCoder]</a>} | || |^{57.3 pass@1} | ^{<a href="https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement" target="_blank">OpenRAIL-M</a>} |
</font>

示例代码：
python
from modelscope import MsDataset
from modelscope.utils.constant import DownloadMode

ds = MsDataset.load('AI-ModelScope/WizardLM_evol_instruct_V2_196k',subset_name='default', split='train', download_mode=DownloadMode.FORCE_REDOWNLOAD)
print(next(iter(ds)))

**仓库地址**：https://github.com/nlpxucan/WizardLM
**Twitter账号**：https://twitter.com/WizardLM_AI/status/1669364947606982656

**该数据集包含143K条由Alpaca与ShareGPT演化而来的混合训练数据**。
**这是WizardLM模型Evol-Instruct训练数据的最新优化版本**。
**由于数据使用授权协议限制，请将原始[ShareGPT](https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered)数据集与本数据集合并，以获得最终完整数据集，该数据集总计约包含196K条数据样本**。