MINT-SJTU/STI-Bench

STI-Bench是一个用于评估多模态大型语言模型（MLLMs）对现实世界视频数据中空间时间概念理解能力的基准数据集。该数据集包含超过2000个问题-答案对，涵盖300个视频，这些视频来源于Omni6DPose、ScanNet和Waymo等数据集，包含了桌面设置、室内场景和室外场景等真实世界环境。STI-Bench旨在挑战模型在静态和动态空间时间任务上的能力，包括3D视频定位、自我中心定位、姿态估计、尺寸测量、位移和路径长度估计、速度和加速度预测、空间关系识别和轨迹描述等任务。

STI-Bench is a benchmark dataset for evaluating the ability of Multimodal Large Language Models (MLLMs) to understand spatial-temporal concepts through real-world video data. The dataset contains more than 2,000 question-answer pairs across 300 videos, sourced from datasets like Omni6DPose, ScanNet, and Waymo, covering real-world environments such as desktop settings, indoor scenes, and outdoor scenarios. STI-Bench is designed to challenge models on both static and dynamic spatial-temporal tasks, including 3D Video Grounding, Ego-Centric Orientation, Pose Estimation, Dimensional Measurement, Displacement & Path Length, Speed & Acceleration, Spatial Relation, and Trajectory Description tasks.