cmudrc/OpenSeeSimE-Structural

OpenSeeSimE-Structural是一个大规模基准数据集，用于评估视觉语言模型在结构分析模拟解释任务上的性能。数据集包含超过100,000个问答对，涵盖了参数化变化的结构模拟，包括应力分析和变形模式。数据集的主要目的是评估视觉语言模型在工程可视化上的性能，包括多种推理能力（如描述、推理、基础理解、关系理解）和不同的问题类型（如二元分类、多项选择、空间基础）。数据集由102,678个实例组成，包含5种结构模型和1,024种参数组合。数据集的生成过程包括模拟生成和真实值提取，预处理和数据格式也进行了详细描述。数据集的用途主要是作为基准评估和能力评估，但也指出了使用中的限制和伦理考虑。

OpenSeeSimE-Structural is a large-scale benchmark dataset for evaluating vision-language models on structural analysis simulation interpretation tasks. It contains over 100,000 question-answer pairs across parametrically-varied structural simulations including stress analysis, and deformation patterns. The purpose of this benchmark is to enable statistically robust evaluation of VLM performance on engineering visualizations, assessment across multiple reasoning capabilities (captioning, reasoning, grounding, relationship understanding), and evaluation using different question types (binary classification, multiple-choice, spatial grounding). The dataset consists of 102,678 instances, including 5 structural models and 1,024 parameter combinations. The dataset generation process includes simulation generation and ground truth extraction, with detailed descriptions of preprocessing and data formats. The primary use cases are benchmark evaluation and capability assessment, but it also notes limitations and ethical considerations in usage.