opendiffusionai/cc12m-cleaned

该数据集基于Conceptual Captions 12million数据集和CaptionEmporium的LLaVa标注子集构建。通过使用LLaVa标注作为基础，进一步过滤掉包含水印、艺术家签名等问题的图像，并手动删除了所有来自dreamstime.com等带有水印的图像。最终的数据集主要去除了图像中的字幕、水印等可能影响AI图像模型训练的因素。数据集目前包含约850万张图像，并以JSONL格式提供。下载图像时，提供了一个示例脚本crawl.sh，并建议用户根据需求调整脚本以排除极端宽高比的图像。此外，数据集还移除了被认为对训练有干扰的图像，如拼贴画和分屏图像。

This dataset is built on the Conceptual Captions 12million dataset and the LLaVa captioned subset by CaptionEmporium. It has been filtered to remove images with watermarks, artist signatures, etc., resulting in a dataset of approximately 8.5 million images, primarily for training AI image models to avoid image interference. The dataset is stored in JSONL format and includes a sample script for downloading images.