SpaCE-10

SpaCE-10是一个全面的基准测试，用于评估多模态大型语言模型在组合空间智能方面的能力。该数据集由上海交通大学、上海人工智能实验室、厦门大学、中山大学和新加坡国立大学的研究人员创建。SpaCE-10包含超过5,000个高质量的问答对，涵盖811个真实室内场景，并覆盖了多种评估设置，如点云输入和多选问答。数据集的创建过程涉及超过150小时的人类专家工作，旨在通过创新的分层标注流程生成高质量的多样化问答对。SpaCE-10旨在解决现有基准测试在评估多模态大型语言模型的空间智能方面的不足，并推动该领域的研究和发展。

SpaCE-10 is a comprehensive benchmark for evaluating the compositional spatial intelligence capabilities of multimodal large language models. This dataset was developed by researchers from Shanghai Jiao Tong University, Shanghai AI Laboratory, Xiamen University, Sun Yat-sen University, and the National University of Singapore. SpaCE-10 contains over 5,000 high-quality question-answer pairs, covering 811 real indoor scenes, and supports multiple evaluation configurations including point cloud input and multiple-choice question answering. The dataset creation process involved more than 150 hours of human expert effort, aiming to generate high-quality and diverse question-answer pairs via an innovative hierarchical annotation pipeline. SpaCE-10 is designed to address the gaps in existing benchmarks for evaluating the spatial intelligence of multimodal large language models, and to facilitate research and advancement in this domain.