VideoNavQA
收藏资源简介：
该数据集名为VideoNavQA，包含了在House3D环境中生成的问答对，这些问题与视频相结合，旨在评估从近乎理想的导航路径中回答问题的性能。该数据集覆盖了28种问题类型，分为8个类别，并配有70个可能的答案。每种问题类型都与一个用于程序化生成的模板相关联。该数据集大约包含了101,000对视频和问题的组合，其任务类型为具身问答（Embodied Question Answering，简称Eqa）。
The dataset named VideoNavQA consists of question-answer pairs generated in the House3D environment, paired with corresponding videos, and is designed to evaluate the performance of question answering systems that utilize environmental information gathered from near-ideal navigation paths. It includes 28 question types grouped into 8 categories, with 70 possible answer options available. Each question type is associated with a template for procedural generation. The dataset contains approximately 101,000 video-question pairs, and its task type is Embodied Question Answering (abbreviated as EqA).
VideoNavQA 数据集概述
数据集简介
- 名称：VideoNavQA
- 任务：视觉推理任务，专注于回答与视频内容相关的问题，但不涉及导航和动作选择。
- 目标：提高视觉推理的难度，处理更复杂的推理问题。
- 相关论文：VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering (BMVC 2019, ViGIL NeurIPS 2019)
数据集样例
-
-
-
-
样例问题：
- Where is the green rug next to the sofa?
- Are the computer and the bed the same color?
- What is the thing next to the tv stand located in the living room?
数据集获取
bash $ git clone https://github.com/catalina17/VideoNavQA $ virtualenv -p python3 videonavqa $ source videonavqa/bin/activate $ pip install -r requirements.txt
数据集链接
数据集统计
依赖
- 模型评估：
- Faster-RCNN (使用VGG-16预训练权重)
- 预训练对象检测器（从指定checkpoint加载）
- 数据生成工具：
- EmbodiedQA fork
- House3D fork
- SUNCG dataset
- SUNCG toolbox
模型运行
- 使用
eval.sh脚本运行FiLM-based模型 - 使用
q_and_v_test.py脚本进行模型测试
引用
@article{cangea2019videonavqa, title={VideoNavQA: Bridging the Gap between Visual and Embodied Question Answering}, author={Cangea, C{u{a}}t{u{a}}lina and Belilovsky, Eugene and Li{`o}, Pietro and Courville, Aaron}, journal={arXiv preprint arXiv:1908.04950}, year={2019} }
