EleutherAI/deep-ignorance-annealing-mix

Deep Ignorance模型套件是一系列用于研究预训练数据过滤对语言模型能力影响，特别是防止学习不安全技术知识（如CBRN）的模型。数据集中包含在不同的预训练和退火阶段应用了不同过滤技术的各种模型。这些模型旨在进行研究和不是用于部署，并且它们可以在HuggingFace上用于训练和推理。

The Deep Ignorance Model Suite is a collection of models designed to explore the effects of filtering pretraining data on language model capabilities, especially in relation to preventing the learning of unsafe technical knowledge. The dataset includes various models with different filtering techniques applied during pretraining and annealing stages. The models are intended for research and not for deployment, and they are available on HuggingFace for training and inference.