cygu/sampling-distill-train-data-kth-shift1

---
dataset_info:
features:
- name: text
dtype: string
splits:
- name: train
num_bytes: 634109141
num_examples: 640000
download_size: 430041654
dataset_size: 634109141
---
# Dataset Card for "sampling-distill-train-data-kth-shift1"

Training data for sampling-based watermark distillation using the KTH \\(s=1\\) watermarking strategy in the paper [On the Learnability of Watermarks for Language Models](https://arxiv.org/abs/2312.04469). [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) with decoding-based watermarking was used to generate 640,000 watermarked samples, each 256 tokens long. Each sample is prompted with 50-token prefixes from [OpenWebText](https://huggingface.co/datasets/Skylion007/openwebtext) (prompts not included in the samples).

原始信息汇总

数据集概述

数据集名称

• 名称: sampling-distill-train-data-kth-shift1

数据集特征

• 特征:
  • text: 数据类型为字符串。

数据集拆分

• 拆分:
  • train:
    • 示例数量: 640000
    • 数据大小: 634109141字节

数据集大小

• 下载大小: 430041654字节
• 数据集大小: 634109141字节

数据集生成

• 使用Llama 2 7B模型生成640,000个水印样本，每个样本256个令牌长。每个样本前缀为来自OpenWebText的50个令牌（前缀不包含在样本中）。