cygu/sampling-distill-train-data-kgw-k2-gamma0.25-delta2

---
dataset_info:
features:
- name: text
dtype: string
splits:
- name: train
num_bytes: 643139280
num_examples: 640000
download_size: 448465150
dataset_size: 643139280
---
# Dataset Card for "sampling-distill-train-data-kgw-k2-gamma0.25-delta2"

Training data for sampling-based watermark distillation using the KGW \\(k=2, \gamma=0.25, \delta=2\\) watermarking strategy in the paper [On the Learnability of Watermarks for Language Models](https://arxiv.org/abs/2312.04469). [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from [OpenWebText](https://huggingface.co/datasets/Skylion007/openwebtext) (prompts not included in the samples).

原始信息汇总

数据集概述

数据集信息

• 特征:
  • 名称: text
  • 数据类型: string

数据分割

• 训练集:
  • 名称: train
  • 字节数: 643139280
  • 样本数: 640000

数据大小

• 下载大小: 448465150
• 数据集大小: 643139280

数据集描述

• 数据集名称: sampling-distill-train-data-kgw-k2-gamma0.25-delta2
• 生成方法: 使用KGW水印策略（k=2, γ=0.25, δ=2）生成的基于采样的水印蒸馏训练数据。
• 生成模型: Llama 2 7B
• 样本数量: 640000个水印样本
• 样本长度: 每个样本256个token
• 样本前缀: 每个样本包含50个token的前缀，来自OpenWebText数据集（前缀不包含在样本中）