anthracite-org/pixmo-point-explanations-images

PixMo-Point-Explanations是一个包含图像、问题和带有解释的答案的数据集，这些解释可能包括指向图像部分的点。该数据集用于训练视觉语言模型，使其能够通过文本和点的混合方式回答问题。数据集是PixMo数据集集合的一部分，并用于训练Molmo系列模型。数据集被认为是实验性的，虽然这些解释可能非常有用，但我们也观察到模型在生成此类输出时可能会产生更多的幻觉。因此，Molmo模型被训练为仅在输入问题前加上“point_qa:”时生成此类输出。数据集遵循ODC-BY-1.0许可证，适用于研究和教育用途。

PixMo-Point-Explanations is a dataset of images, questions, and answers with explanations that can include in-line points that refer to parts of the image. It can be used to train vision language models to respond to questions through a mixture of text and points. The dataset is part of the PixMo dataset collection and was used to train the Molmo family of models. We consider this dataset experimental, while these explanations can be very informative we have also seen models can hallucinate more when generating outputs of this sort. For that reason, the Molmo models are trained to only generate outputs like this when specifically requested by prefixing input questions with point_qa:. The dataset is licensed under ODC-BY-1.0 and is intended for research and educational use.