OIG

OIG（Open Instruction Generalist）数据集是一个由LAION-AI创建的大型指令数据集，旨在训练能够执行多种任务的助手模型。该数据集包含大量的中等质量指令数据以及较小的高质量指令数据集（OIG-small-chip2），数据以jsonl对象形式存在，至少包含一个text字段，部分数据集还可能包含metadata字段。OIG数据集目前包含4400万个数据点，目标是通过创建1万亿个多样化的指令数据集，足以从头开始预训练大型语言模型。

The OIG (Open Instruction Generalist) dataset is a large-scale instruction dataset created by LAION-AI, designed to train assistant models capable of performing diverse tasks. This dataset includes a large volume of medium-quality instruction data and a smaller high-quality instruction subset named OIG-small-chip2. All data is stored in JSONL objects, which must contain at least a `text` field, and some entries may also include a `metadata` field. Currently, the OIG dataset contains 44 million data points. Its ultimate goal is to build a diversified instruction dataset of 1 trillion samples, which is sufficient for pre-training large language models from scratch.

搜集汇总

背景与挑战

背景概述

OIG数据集是一个包含约4300万条指令的大规模开源数据集，旨在通过多样化的数据源和对话格式支持指令跟随模型的开发。它覆盖了从学术问题到创意写作和编程的广泛任务，目标是帮助将预训练语言模型转化为能够遵循指令的模型。