MIRA-SJTU/STI-Bench

STI-Bench是一个用于评估多模态大型语言模型（MLLMs）对现实世界视频数据中空间时间概念理解能力的数据集。它包含了超过2000个问题答案对，跨越了300个视频，这些视频来源于Omni6DPose、ScanNet和Waymo等数据集。数据集设计用于挑战模型在静态和动态空间时间任务上的能力，包括3D视频定位、自我中心定位、姿态估计、尺寸测量、位移与路径长度估计、速度与加速度预测、空间关系识别和轨迹描述等任务。

STI-Bench is a dataset designed to evaluate the ability of Multimodal Large Language Models (MLLMs) to understand spatial-temporal concepts from real-world video data. It contains more than 2,000 question-answer pairs across 300 videos sourced from datasets like Omni6DPose, ScanNet, and Waymo. The dataset is designed to challenge models on both static and dynamic spatial-temporal tasks, including 3D Video Grounding, Ego-Centric Orientation, Pose Estimation, Dimensional Measurement, Displacement & Path Length, Speed & Acceleration, Spatial Relation, and Trajectory Description.