Molmo2-VideoCountEval

# Molmo2-VideoCountEval Molmo2-VideoCountEval is a validation set of video pointing data collected from human annotators. It can be used to evaluate vision-language models' video counting capability. Molmo2-VideoCountEval is part of the [Molmo2 dataset collection](https://huggingface.co/collections/allenai/molmo2-data) and was used to evaluate the [Molmo2 family of models](https://huggingface.co/collections/allenai/molmo2). Quick links: - 📃 [Paper](https://allenai.org/papers/molmo2) - 🎥 [Blog with Videos](https://allenai.org/blog/molmo2) ## Data Format - `video_source`: There are two video sources: `youtube`, and `MammalNet`. For YouTube videos, you need to download them by their IDs. We provide a mapping from their IDs to the original YouTube URLs and public Google Cloud Storage URLs in `youtube_id_to_urls_mapping.json`. For videos from MammalNet, you can download them following the instructions in their Github repo [here](https://github.com/Vision-CAIR/MammalNet?tab=readme-ov-file#dataset-download). - `clip_start` and `clip_end`: These represent the randomly sampled start and end times we used to clip the videos with a `video_duration` > 63 seconds to fit max 128 frames at 2FPS in our evaluation. ## License This dataset is licensed under ODC-BY. A subset of videos from this dataset that are licensed as CC BY-4.0 may be downloaded from our Google Cloud Bucket via the URLs in `youtube_id_to_urls_mapping.json`. The dataset and videos are intended for research and educational use in accordance with Ai2’s [Responsible Use Guidelines](https://allenai.org/responsible-use). This dataset includes questions generated from GPT-4.1 and GPT-5, which are subject to OpenAI’s [Terms of Use](https://openai.com/policies/row-terms-of-use/).