sdffdxsf-vze1/databricks-dolly-15k

--- license: cc-by-sa-3.0 task_categories: - question-answering - summarization language: - en size_categories: - 10K<n<100K --- # Summary `databricks-dolly-15k` is an open source dataset of instruction-following records generated by thousands of Databricks employees in several of the behavioral categories outlined in the [InstructGPT](https://arxiv.org/abs/2203.02155) paper, including brainstorming, classification, closed QA, generation, information extraction, open QA, and summarization. This dataset can be used for any purpose, whether academic or commercial, under the terms of the [Creative Commons Attribution-ShareAlike 3.0 Unported License](https://creativecommons.org/licenses/by-sa/3.0/legalcode). Supported Tasks: - Training LLMs - Synthetic Data Generation - Data Augmentation Languages: English Version: 1.0 **Owner: Databricks, Inc.** # Dataset Overview `databricks-dolly-15k` is a corpus of more than 15,000 records generated by thousands of Databricks employees to enable large language models to exhibit the magical interactivity of ChatGPT. Databricks employees were invited to create prompt / response pairs in each of eight different instruction categories, including the seven outlined in the InstructGPT paper, as well as an open-ended free-form category. The contributors were instructed to avoid using information from any source on the web with the exception of Wikipedia (for particular subsets of instruction categories), and explicitly instructed to avoid using generative AI in formulating instructions or responses. Examples of each behavior were provided to motivate the types of questions and instructions appropriate to each category. Halfway through the data generation process, contributors were given the option of answering questions posed by other contributors. They were asked to rephrase the original question and only select questions they could be reasonably expected to answer correctly. For certain categories contributors were asked to provide reference texts copied from Wikipedia. Reference text (indicated by the `context` field in the actual dataset) may contain bracketed Wikipedia citation numbers (e.g. `[42]`) which we recommend users remove for downstream applications. # Intended Uses While immediately valuable for instruction fine tuning large language models, as a corpus of human-generated instruction prompts, this dataset also presents a valuable opportunity for synthetic data generation in the methods outlined in the Self-Instruct paper. For example, contributor--generated prompts could be submitted as few-shot examples to a large open language model to generate a corpus of millions of examples of instructions in each of the respective InstructGPT categories. Likewise, both the instructions and responses present fertile ground for data augmentation. A paraphrasing model might be used to restate each prompt or short responses, with the resulting text associated to the respective ground-truth sample. Such an approach might provide a form of regularization on the dataset that could allow for more robust instruction-following behavior in models derived from these synthetic datasets. # Dataset ## Purpose of Collection As part of our continuing commitment to open source, Databricks developed what is, to the best of our knowledge, the first open source, human-generated instruction corpus specifically designed to enable large language models to exhibit the magical interactivity of ChatGPT. Unlike other datasets that are limited to non-commercial use, this dataset can be used, modified, and extended for any purpose, including academic or commercial applications. ## Sources - **Human-generated data**: Databricks employees were invited to create prompt / response pairs in each of eight different instruction categories. - **Wikipedia**: For instruction categories that require an annotator to consult a reference text (information extraction, closed QA, summarization) contributors selected passages from Wikipedia for particular subsets of instruction categories. No guidance was given to annotators as to how to select the target passages. ## Annotator Guidelines To create a record, employees were given a brief description of the annotation task as well as examples of the types of prompts typical of each annotation task. Guidelines were succinct by design so as to encourage a high task completion rate, possibly at the cost of rigorous compliance to an annotation rubric that concretely and reliably operationalizes the specific task. Caveat emptor. The annotation guidelines for each of the categories are as follows: - **Creative Writing**: Write a question or instruction that requires a creative, open-ended written response. The instruction should be reasonable to ask of a person with general world knowledge and should not require searching. In this task, your prompt should give very specific instructions to follow. Constraints, instructions, guidelines, or requirements all work, and the more of them the better. - **Closed QA**: Write a question or instruction that requires factually correct response based on a passage of text from Wikipedia. The question can be complex and can involve human-level reasoning capabilities, but should not require special knowledge. To create a question for this task include both the text of the question as well as the reference text in the form. - **Open QA**: Write a question that can be answered using general world knowledge or at most a single search. This task asks for opinions and facts about the world at large and does not provide any reference text for consultation. - **Summarization**: Give a summary of a paragraph from Wikipedia. Please don't ask questions that will require more than 3-5 minutes to answer. To create a question for this task include both the text of the question as well as the reference text in the form. - **Information Extraction**: These questions involve reading a paragraph from Wikipedia and extracting information from the passage. Everything required to produce an answer (e.g. a list, keywords etc) should be included in the passages. To create a question for this task include both the text of the question as well as the reference text in the form. - **Classification**: These prompts contain lists or examples of entities to be classified, e.g. movie reviews, products, etc. In this task the text or list of entities under consideration is contained in the prompt (e.g. there is no reference text.). You can choose any categories for classification you like, the more diverse the better. - **Brainstorming**: Think up lots of examples in response to a question asking to brainstorm ideas. ## Personal or Sensitive Data This dataset contains public information (e.g., some information from Wikipedia). To our knowledge, there are no private person’s personal identifiers or sensitive information. ## Language American English # Known Limitations - Wikipedia is a crowdsourced corpus and the contents of this dataset may reflect the bias, factual errors and topical focus found in Wikipedia - Some annotators may not be native English speakers - Annotator demographics and subject matter may reflect the makeup of Databricks employees # Citation ``` @online{DatabricksBlog2023DollyV2, author = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin}, title = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM}, year = {2023}, url = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm}, urldate = {2023-06-30} } ``` # License/Attribution **Copyright (2023) Databricks, Inc.** This dataset was developed at Databricks (https://www.databricks.com) and its use is subject to the CC BY-SA 3.0 license. Certain categories of material in the dataset include materials from the following sources, licensed under the CC BY-SA 3.0 license: Wikipedia (various pages) - https://www.wikipedia.org/ Copyright © Wikipedia editors and contributors.许可证：CC BY-SA 3.0 任务类别： - 问答 - 摘要 语言： - 英语 样本量范围：10000 < 样本量 < 100000 --- # 摘要 `databricks-dolly-15k` 是一个开源的指令遵循记录数据集，由数千名Databricks员工基于[InstructGPT](https://arxiv.org/abs/2203.02155)论文中列出的多个行为类别生成，涵盖头脑风暴、分类、封闭域问答、生成、信息抽取、开放域问答以及摘要任务。 本数据集可在[知识共享署名-相同方式共享3.0未移植许可协议](https://creativecommons.org/licenses/by-sa/3.0/legalcode)条款下用于学术或商业等任意用途。 支持的任务： - 大语言模型（Large Language Model，LLM）训练 - 合成数据生成 - 数据增强 语言：英语 版本：1.0 **所有者：Databricks公司** # 数据集概览 `databricks-dolly-15k` 是一个包含超15000条记录的语料库，由数千名Databricks员工创建，旨在让大语言模型（Large Language Model，LLM）具备类似ChatGPT的出色交互能力。 Databricks员工受邀在8种不同的指令类别中创建提示词-应答对，其中包括InstructGPT论文中提及的7种类别，以及一个开放式自由形式类别。参与者被要求除特定指令类别可使用维基百科内容外，不得使用网络上的其他任何来源信息，且明确禁止在生成指令或应答时使用生成式AI。每个行为类别的示例均已提供，以引导参与者生成符合各分类要求的问题与指令。 在数据生成过程进行到一半时，参与者被允许回答其他参与者提出的问题，且需重新表述原问题，并仅选择自身有能力准确作答的问题。 对于部分类别，参与者需提供源自维基百科的参考文本。数据集中的`context`字段即代表参考文本，其中可能包含带方括号的维基百科引用编号（如`[42]`），建议下游应用使用者将其移除。 # 预期用途 本数据集作为人工生成的指令提示词语料库，不仅可直接用于大语言模型的指令微调，还为采用《Self-Instruct》论文中所述方法进行合成数据生成提供了宝贵契机。例如，可将参与者生成的提示词作为少样本（Few-shot）示例，输入至开源大语言模型中，为InstructGPT的各分类生成数百万条指令示例语料库。 同理，指令与应答本身均适合用于数据增强：可借助释义模型对每条提示词或简短应答进行改写，将生成的文本与对应的真实标注样本关联。此类方法可实现数据集的正则化，使基于此类合成数据集训练的模型能够具备更稳健的指令遵循能力。 # 数据集 ## 收集目的 作为我们对开源社区持续承诺的一部分，Databricks开发了目前已知的首个开源、人工生成的指令语料库，专门用于赋能大语言模型实现类似ChatGPT的出色交互能力。与其他仅限非商业用途的数据集不同，本数据集可用于、修改并扩展至任意场景，包括学术或商业应用。 ## 数据来源 - **人工生成数据**：Databricks员工受邀在8种不同的指令类别中创建提示词-应答对。 - **维基百科**：对于需要标注员参考文本的指令类别（信息抽取、封闭域问答、摘要），参与者可从维基百科中选取对应类别的段落。未向标注员提供选取目标段落的具体指导。 ## 标注指南 为创建记录，参与者会收到标注任务的简要说明以及各标注任务的典型提示词示例。标注指南设计得简洁明了，以鼓励较高的任务完成率，但可能导致标注员未能严格遵循能够精准且可靠定义特定任务的标注细则。特此提示。 各分类的标注指南如下： - **创意写作**：编写需要创意性、开放式书面应答的问题或指令。该指令应可通过具备通用世界知识的普通人完成，无需额外检索。本任务中，提示词应给出具体的遵循要求，约束、说明、指南或要求均可，且越详细越好。 - **封闭域问答**：编写需要基于维基百科文本段落给出准确事实性应答的问题或指令。问题可较为复杂，需具备人类级别的推理能力，但无需专业知识。创建本任务的问题时，需同时包含问题文本与参考文本。 - **开放域问答**：编写可通过通用世界知识或至多一次搜索即可作答的问题。本任务要求回答关于世界的观点与事实，无需提供参考文本。 - **摘要**：对维基百科的段落进行总结。请勿提出需要超过3-5分钟作答的问题。创建本任务的问题时，需同时包含问题文本与参考文本。 - **信息抽取**：此类问题要求阅读维基百科段落并从中提取信息。生成答案所需的全部内容（如列表、关键词等）均应包含在段落中。创建本任务的问题时，需同时包含问题文本与参考文本。 - **分类**：此类提示词包含待分类的实体列表或示例，例如影评、产品等。本任务中，待分类的文本或实体列表需包含在提示词中（无需参考文本）。可任意选择分类类别，多样性越高越好。 - **头脑风暴**：针对要求构思想法的问题，生成大量示例。 ## 个人或敏感数据 本数据集包含公开信息（例如维基百科的部分内容）。据我们所知，数据集中未包含任何个人身份信息或敏感内容。 ## 语言 美式英语 # 已知局限性 - 维基百科是众包语料库，本数据集的内容可能反映维基百科中存在的偏见、事实错误与主题聚焦倾向。 - 部分标注者并非以英语为母语。 - 标注者的人口统计特征与主题选择可能反映Databricks员工的构成情况。 # 引用 @online{DatabricksBlog2023DollyV2, author = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin}, title = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM}, year = {2023}, url = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm}, urldate = {2023-06-30} } # 许可证与署名 **版权所有（2023）Databricks公司** 本数据集由Databricks（https://www.databricks.com）开发，使用需遵循CC BY-SA 3.0许可证。 数据集中部分类别内容源自以下来源，这些内容已获得CC BY-SA 3.0许可证授权： 维基百科（各页面）- https://www.wikipedia.org/ 版权归维基百科编辑者与贡献者所有。