OpenEQA

该数据集为评估基于结构化场景表示的具身问答系统性能提供了一个基准。它包含了来自3D扫描的各种问题和关键帧，这使得我们能够评估语言引导下的场景图更新能力。该数据集还提供了情景记忆变体，其中包含固定关键帧，任务专注于具身问答。This dataset serves as a benchmark for evaluating the performance of embodied question answering systems based on structured scene representations. It contains various questions and keyframes sourced from 3D scans, enabling the evaluation of language-guided scene graph update capabilities. The dataset also provides episodic memory variants, which include fixed keyframes and have tasks focused on embodied question answering.

搜集汇总

背景与挑战

背景概述

OpenEQA是一个开放词汇的体现问题回答基准测试数据集，包含1600多个非模板化问题，覆盖180多个真实环境，支持情景记忆和主动探索两种用例，并提供了基于LLM的自动评估协议。