Junaid7188/text-to-image-2M

text-to-image-2M是一个用于微调文本到图像模型的高质量、多样化的文本-图像对数据集。该数据集包含约200万个样本，经过精心选择和增强，以满足文本到图像模型训练的高要求。其创建动机源于观察到超过100万样本的数据集往往能产生更好的微调效果，但现有公开数据集常存在图像质量无保证、类别不平衡、缺乏多样性或规模不适宜等问题。为此，该数据集结合并增强了多个现有高质量数据集，使用先进的文本到图像和字幕模型（如Qwen2-VL、Flux-dev、GPT-4o等）构建，包括data_512_2M（200万个512x512图像微调数据集）和data_1024_10K（1万个高质量高分辨率图像数据集，用于高分辨率适应）。数据集来源包括LLaVA-next微调数据集（约70万样本，使用Qwen2-VL重新标注）、LLaVA预训练数据集（约50万样本，原始提示，Flux-dev生成图像）、ProGamerGov合成数据集（约90万样本，经有效性过滤）和GPT-4o生成数据集（10万样本）。数据集采用WebDataset格式，可通过HuggingFace的datasets库轻松访问。

text-to-image-2M is a curated text-image pair dataset designed for fine-tuning text-to-image models. The dataset consists of approximately 2 million samples, carefully selected and enhanced to meet the high demands of text-to-image model training. The motivation behind creating this dataset stems from the observation that datasets with over 1 million samples tend to produce better fine-tuning results, but existing publicly available datasets often have limitations such as unguaranteed image quality, lack of category balance or diversity, and size constraints. To address these issues, the dataset combines and enhances existing high-quality datasets using state-of-the-art text-to-image and captioning models (e.g., Qwen2-VL, Flux-dev, GPT-4o). It includes data_512_2M, a 2M 512x512 fine-tuning dataset, and data_1024_10K, a 10K high-quality, high-resolution dataset for high-resolution adaptation. The dataset is composed of subsets from LLaVA-next fine-tuning dataset (~700K samples, re-captioned using Qwen2-VL), LLaVA-pretrain dataset (~500K samples, original prompts with images generated by Flux-dev), ProGamerGov synthetic dataset (~900K samples, filtered for validity), and GPT-4o generated dataset (100K samples). The dataset uses the WebDataset format and can be easily accessed via HuggingFaces datasets library.