A Benchmark for Evaluating Large Language Models in Structural Engineering

This record contains Release 1.0 of an English-normalized benchmark for evaluating large language models in structural engineering.

The benchmark contains 300 base tasks and 800 executable instances organized into three difficulty levels: foundational, intermediate, and advanced. The tasks cover equilibrium and internal actions, trusses, section stresses, reinforced concrete, instability and second-order effects, influence lines, structural dynamics, wave and site-response reasoning, torsion and shear-centre interpretation, structural health monitoring, and post-earthquake judgement.

本记录收录了用于评估结构工程领域大语言模型（Large Language Model，LLM）的英语标准化基准测试集的1.0版本。

该基准测试集包含300项基础任务与800个可执行实例，分为三个难度层级：基础级、中级与高级。任务覆盖平衡与内力分析、桁架结构、截面应力、钢筋混凝土结构、失稳与二阶效应、影响线、结构动力学、波浪与场地响应推理、扭转与剪切中心解析、结构健康监测以及震后评估等范畴。

本次发布包含基准测试集的CSV文件、便捷划分集、提示词构建脚本、评分与分析脚本、列结构定义、带文件哈希值的清单文件、数据集卡片、引用元数据以及许可信息。

该基准测试集具备可溯源性且采用闭式解法，但并非隐藏式或防污染测试集。其设计初衷是作为可复现的研究成果载体，用于在结构工程领域对大语言模型性能开展基准条件下的评估，评估维度涵盖准确率、单位一致性、按严重程度加权的误差、置信度校准以及缺失信息拒绝行为。