GINC(Generative IN-Context learning Dataset)

GINC（Generative In-Context learning Dataset）是一个用于研究上下文学习的小规模合成数据集。预训练数据由 HMM 的混合生成，上下文学习提示示例也由 HMM 生成（无论是否来自混合）。提示示例相对于预训练数据是不分布的，因为每个示例都是独立的、连接的并由分隔符分隔。 GitHub 存储库提供代码来生成具有不同词汇量、HMM 数量和其他参数的 GINC 样式数据集。GINC (Generative In-Context Learning Dataset) is a small-scale synthetic dataset designed for in-context learning research. Its pre-trained data is generated by a mixture of Hidden Markov Models (HMMs), and the in-context learning prompt examples are also generated by HMMs, regardless of whether they originate from the mixture. The prompt examples are distributionally misaligned with the pre-trained data, as each individual example is independent, and the examples are concatenated and separated by delimiters. The GitHub repository provides code for generating GINC-style datasets with varying vocabulary sizes, numbers of HMMs, and other configurable parameters.

搜集汇总

背景与挑战

背景概述

GINC是一个用于研究上下文学习的小规模合成数据集，其数据通过隐马尔可夫模型生成。该数据集的设计特点在于提示示例相对于预训练数据不分布，并提供了生成不同参数数据集的代码。