toloka/WSDMCup2023

--- annotations_creators: - crowdsourced language: - en language_creators: - crowdsourced license: - cc-by-4.0 multilinguality: - monolingual pretty_name: WSDMCup2023 size_categories: - 10K<n<100K source_datasets: [] tags: - toloka task_categories: - visual-question-answering task_ids: - visual-question-answering dataset_info: features: - name: image dtype: string - name: width dtype: int64 - name: height dtype: int64 - name: left dtype: int64 - name: top dtype: int64 - name: right dtype: int64 - name: bottom dtype: int64 - name: question dtype: string splits: - name: train num_examples: 38990 - name: train_sample num_examples: 1000 - name: test_public num_examples: 1705 - name: test_private num_examples: 4504 config_name: wsdmcup2023 --- # Dataset Card for WSDMCup2023 ## Dataset Description - **Homepage:** [Toloka Visual Question Answering Challenge](https://toloka.ai/challenges/wsdm2023) - **Repository:** [WSDM Cup 2023 Starter Pack](https://github.com/Toloka/WSDMCup2023) - **Paper:** <https://arxiv.org/abs/2309.16511> - **Leaderboard:** [CodaLab Competition Leaderboard](https://codalab.lisn.upsaclay.fr/competitions/7434#results) - **Point of Contact:** [e-mail] | Question | Image and Answer | | --- | --- | | What do you use to hit the ball? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/KUsGAc_eqdMcNxkBXzzl/KUsGAc_eqdMcNxkBXzzl_webp_1280_x2.webp" width="228" alt="What do you use to hit the ball?"> | | What do people use for cutting? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/brXEVYckNLfQKcfNu4DF/brXEVYckNLfQKcfNu4DF_webp_1280_x2.webp" width="228" alt="What do people use for cutting?"> | | What do we use to support the immune system and get vitamin C? | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/HQ0A-ZvZCGCmYfTs83K7/HQ0A-ZvZCGCmYfTs83K7_webp_1280_x2.webp" width="228" alt="What do we use to support the immune system and get vitamin C?"> | ### Dataset Summary The WSDMCup2023 Dataset consists of images associated with textual questions. One entry (instance) in our dataset is a question-image pair labeled with the ground truth coordinates of a bounding box containing the visual answer to the given question. The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in Context dataset, [MS COCO](https://cocodataset.org/). All data labeling was performed on the [Toloka crowdsourcing platform](https://toloka.ai/). Our dataset has 45,199 instances split among three subsets: train (38,990 instances), public test (1,705 instances), and private test (4,504 instances). The entire train dataset was available for everyone since the start of the challenge. The public test dataset was available since the evaluation phase of the competition but without any ground truth labels. After the end of the competition, public and private sets were released. ## Dataset Citation Please cite the challenge results or dataset description as follows. - Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. [Toloka Visual Question Answering Benchmark](https://arxiv.org/abs/2309.16511). 2023. arXiv: [2309.16511 [cs.CV]](https://arxiv.org/abs/2309.16511). ```bibtex @inproceedings{TolokaWSDMCup2023, author = {Ustalov, Dmitry and Pavlichenko, Nikita and Koshelev, Sergey and Likhobaba, Daniil and Smirnova, Alisa}, title = {{Toloka Visual Question Answering Benchmark}}, year = {2023}, eprint = {2309.16511}, eprinttype = {arxiv}, eprintclass = {cs.CV}, language = {english}, } ``` ### Supported Tasks and Leaderboards Grounding Visual Question Answering ### Language English ## Dataset Structure ### Data Instances A data instance contains a URL to the picture, information about the image size - width and height, information about the ground truth bounding box - left top and right bottom dots, and contains the question related to the picture. ``` {'image': https://toloka-cdn.azureedge.net/wsdmcup2023/000000000013.jpg, 'width': 640, 'height': 427, 'left': 129, 'top': 192, 'right': 155, 'bottom': 212, 'question': What does it use to breath?} ``` ### Data Fields * image: contains URL to the image * width: value in pixels of image width * height: value in pixels of image height * left: the x coordinate in pixels to determine the left-top dot of the bounding box * top: the y coordinate in pixels to determine the left-top dot of the bounding box * right: the x coordinate in pixels to determine the right-bottom dot of the bounding box * bottom: the y coordinate in pixels to determine the right-bottom dot of the bounding box * question: a question related to the picture ### Data Splits There are four splits in the data: train, train_sample, test_public, and test_private. 'train' split contains the full pull for model training. The 'train-sample' split contains the part of the 'train' split. The 'test_public' split contains public data to test the model. The 'test_private' split contains private data for the final model test. ### Source Data The images were obtained from a CC BY-licensed subset of the Microsoft Common Objects in Context dataset, [MS COCO](https://cocodataset.org/). ### Annotations All data labeling was performed on the [Toloka crowdsourcing platform](https://toloka.ai/). Only annotators who self-reported the knowledge of English had access to the annotation task. ### Citation Information * Competition: https://toloka.ai/challenges/wsdm2023 * CodaLab: https://codalab.lisn.upsaclay.fr/competitions/7434 * Dataset: https://doi.org/10.5281/zenodo.7057740annotations_creators（标注创建者）: - 众包（crowdsourced） language（语言）: - 英语（en） language_creators（语言创建者）: - 众包（crowdsourced） license（许可协议）: - CC BY 4.0协议 multilinguality（多语言属性）: - 单语言（monolingual） pretty_name（易读名称）: WSDMCup2023 size_categories（样本量类别）: - 10K<n<100K source_datasets（源数据集）: [] tags（标签）: - Toloka平台 task_categories（任务类别）: - 视觉问答（Visual Question Answering） task_ids（任务标识）: - 视觉问答（Visual Question Answering） dataset_info（数据集信息）: features（特征）: - name: 图像（image） dtype: 字符串（string） - name: 宽度（width） dtype: 64位整数（int64） - name: 高度（height） dtype: 64位整数（int64） - name: 左边界（left） dtype: 64位整数（int64） - name: 上边界（top） dtype: 64位整数（int64） - name: 右边界（right） dtype: 64位整数（int64） - name: 下边界（bottom） dtype: 64位整数（int64） - name: 问题（question） dtype: 字符串（string） splits（划分）: - name: 训练集（train） num_examples（样本数量）: 38990 - name: 训练采样集（train_sample） num_examples（样本数量）: 1000 - name: 公开测试集（test_public） num_examples（样本数量）: 1705 - name: 私有测试集（test_private） num_examples（样本数量）: 4504 config_name（配置名称）: wsdmcup2023 --- # WSDMCup2023数据集卡片 ## 数据集描述 - **主页**：[Toloka视觉问答挑战赛](https://toloka.ai/challenges/wsdm2023) - **代码仓库**：[WSDM Cup 2023 入门套件](https://github.com/Toloka/WSDMCup2023) - **论文**：<https://arxiv.org/abs/2309.16511> - **排行榜**：[CodaLab竞赛排行榜](https://codalab.lisn.upsaclay.fr/competitions/7434#results) - **联系人**：[e-mail] | 问题 | 图像与答案 | | --- | --- | | 你用什么击打球体？ | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/KUsGAc_eqdMcNxkBXzzl/KUsGAc_eqdMcNxkBXzzl_webp_1280_x2.webp" width="228" alt="What do you use to hit the ball?"> | | 人们用什么进行切割？ | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/brXEVYckNLfQKcfNu4DF/brXEVYckNLfQKcfNu4DF_webp_1280_x2.webp" width="228" alt="What do people use for cutting?"> | | 我们用什么支持免疫系统并获取维生素C？ | <img src="https://tlkfrontprod.azureedge.net/portal-production/static/uploaded/images/HQ0A-ZvZCGCmYfTs83K7/HQ0A-ZvZCGCmYfTs83K7_webp_1280_x2.webp" width="228" alt="What do we use to support the immune system and get vitamin C?"> | ### 数据集摘要 WSDMCup2023数据集由与文本问题关联的图像组成。本数据集的每条数据（实例）均为一组问题-图像对，并标注了包含该问题对应视觉答案的边界框真实坐标。所有图像取自采用知识共享署名许可的微软通用对象在上下文数据集（[MS COCO](https://cocodataset.org/)）子集。全部数据标注工作均在[Toloka众包平台](https://toloka.ai/)完成。 本数据集共包含45199条实例，划分为三个子集：训练集（38990条实例）、公开测试集（1705条实例）与私有测试集（4504条实例）。挑战赛启动之初，完整训练集即对所有参赛者开放。公开测试集于竞赛评估阶段开放，但未附带任何真实标注。竞赛结束后，公开测试集与私有测试集均对外发布。 ## 数据集引用 请按以下方式引用该挑战赛成果或数据集： - Ustalov D., Pavlichenko N., Koshelev S., Likhobaba D., and Smirnova A. [Toloka视觉问答基准数据集](https://arxiv.org/abs/2309.16511). 2023. arXiv: [2309.16511 [cs.CV]](https://arxiv.org/abs/2309.16511). bibtex @inproceedings{TolokaWSDMCup2023, author = {Ustalov, Dmitry and Pavlichenko, Nikita and Koshelev, Sergey and Likhobaba, Daniil and Smirnova, Alisa}, title = {{Toloka视觉问答基准数据集}}, year = {2023}, eprint = {2309.16511}, eprinttype = {arxiv}, eprintclass = {cs.CV}, language = {english}, } ### 支持任务与排行榜 锚定式视觉问答（Grounding Visual Question Answering） ### 语言 英语 ## 数据集结构 ### 数据实例 一条数据实例包含图像的URL、图像尺寸信息（宽度与高度）、真实边界框信息（左上角与右下角坐标），以及与该图像相关的问题。示例如下： {'image': https://toloka-cdn.azureedge.net/wsdmcup2023/000000000013.jpg, 'width': 640, 'height': 427, 'left': 129, 'top': 192, 'right': 155, 'bottom': 212, 'question': 它用什么进行呼吸？} ### 数据字段 * image：图像的URL地址 * width：图像宽度的像素值 * height：图像高度的像素值 * left：用于确定边界框左上角的x轴像素坐标 * top：用于确定边界框左上角的y轴像素坐标 * right：用于确定边界框右下角的x轴像素坐标 * bottom：用于确定边界框右下角的y轴像素坐标 * question：与该图像相关的问题 ### 数据划分 本数据集共包含四个划分：训练集（train）、训练采样集（train_sample）、公开测试集（test_public）与私有测试集（test_private）。其中`train`划分包含用于模型训练的完整数据集；`train_sample`划分包含`train`划分的部分样本；`test_public`划分用于模型的公开测试；`test_private`划分用于模型的最终私密测试。 ### 源数据 所有图像取自采用知识共享署名许可的微软通用对象在上下文数据集（[MS COCO](https://cocodataset.org/)）子集。 ### 标注信息 全部数据标注工作均在[Toloka众包平台](https://toloka.ai/)完成。仅自述掌握英语的标注者可参与标注任务。 ### 引用信息 * 挑战赛：https://toloka.ai/challenges/wsdm2023 * CodaLab平台：https://codalab.lisn.upsaclay.fr/competitions/7434 * 数据集：https://doi.org/10.5281/zenodo.7057740

原始信息汇总

数据集概述

名称: WSDMCup2023

语言: 英语

许可证: CC-BY-4.0

多语言性: 单语种

大小: 10K<n<100K

任务类别: 视觉问答

数据集结构

数据实例

每个数据实例包含以下字段：

• image: 图片的URL
• width: 图片宽度（像素）
• height: 图片高度（像素）
• left: 边界框左上角x坐标（像素）
• top: 边界框左上角y坐标（像素）
• right: 边界框右下角x坐标（像素）
• bottom: 边界框右下角y坐标（像素）
• question: 与图片相关的问题

数据字段

• image: 图片URL
• width: 图片宽度（像素）
• height: 图片高度（像素）
• left: 边界框左上角x坐标（像素）
• top: 边界框左上角y坐标（像素）
• right: 边界框右下角x坐标（像素）
• bottom: 边界框右下角y坐标（像素）
• question: 与图片相关的问题

数据分割

• train: 38,990个实例
• train_sample: 1,000个实例
• test_public: 1,705个实例
• test_private: 4,504个实例

数据来源

图片来源于CC BY-licensed的Microsoft Common Objects in Context (MS COCO) 数据集。

注释信息

所有数据标注在Toloka crowdsourcing平台完成。

引用信息

bibtex @inproceedings{TolokaWSDMCup2023, author = {Ustalov, Dmitry and Pavlichenko, Nikita and Koshelev, Sergey and Likhobaba, Daniil and Smirnova, Alisa}, title = {{Toloka Visual Question Answering Benchmark}}, year = {2023}, eprint = {2309.16511}, eprinttype = {arxiv}, eprintclass = {cs.CV}, language = {english}, }