weijiezz/SpecBlock-train-data-llama
收藏资源简介：
SpecBlock-train-data-llama是一个用于训练SpecBlock草稿模型的数据集，其数据通过从目标模型meta-llama/Llama-3.1-8B-Instruct蒸馏得到。数据准备流程包括：源提示来自ShareGPT（Vicuna未过滤版本）和UltraChat；使用目标模型以贪婪解码方式重新生成每个提示的答案，以获得与目标模型对齐的干净数据分布；随后数据被分割为训练集和评估集。数据集包含两个文件：train.jsonl（训练分割，每行一个对话）和eval.jsonl（评估分割）。该数据集主要用于文本生成任务，支持推测解码和蒸馏相关研究。
Training data for SpecBlock draft models, distilled from the target model meta-llama/Llama-3.1-8B-Instruct. The preparation pipeline involves: source prompts from ShareGPT (Vicuna unfiltered) and UltraChat; re-generating each prompts answer with the target model using greedy decoding to obtain a clean target-aligned distribution; and splitting into training and evaluation splits. The dataset includes files: train.jsonl (training split, one conversation per line) and eval.jsonl (evaluation split). It is designed for text-generation tasks, supporting speculative decoding and distillation research.
