allenai/pixmo-cap

PixMo-Cap是一个包含非常长（平均约200字）且详细描述的数据集，可用于预训练和微调视觉语言模型。该数据集通过记录注释者对图像的描述音频，并使用Claude大型语言模型将音频转录成长描述来创建。数据集还包括音频转录文本。PixMo-Cap是PixMo数据集集合的一部分，并用于训练Molmo系列模型。数据格式包括图像URL、音频转录和LLM生成的描述。数据集遵循ODC-BY-1.0许可证，适用于研究和教育用途。

PixMo-Cap is a dataset of very long (roughly 200 words on average), detailed captions. It can be used to pre-train and fine-tune vision-language models. PixMo-Cap was created by recording annotators speaking about an image for 60-90 seconds and then using the Claude large language model to turn the audio transcripts into a long caption. The audio transcripts are also included. PixMo-Cap is part of the PixMo dataset collection and was used to train the Molmo family of models. The images in the dataset are stored as URLs that will need to be downloaded separately. The dataset is licensed under ODC-BY-1.0, intended for research and educational use.