遇见数据集

Data and code for: Can open language models classify drone operation risk? A step-by-step audit against JARUS SORA 2.5

收藏
Zenodo2026-10-31 更新2026-11-05 收录
官方服务:

资源简介:

Data and code for an audit of ten open-weight large language models (Qwen3 4B/8B/14B/32B-AWQ, OLMo-3-7B-Instruct, Granite-3.3-8B, Phi-4, DeepSeek-R1-Distill-Qwen-14B, gpt-oss-20b) on the JARUS SORA 2.5 Specific Operations Risk Assessment for drone operations. 500 BVLOS operations with exact step answers from a deterministic SORA 2.5 oracle (iGRC, final GRC, initial and residual ARC, TMPR, SAIL, OSO robustness) were posed closed book, with the SORA tables in the prompt, and without naming the edition; 12,600 answers were generated with vLLM 0.30.0 on one A100 80 GB and scored per step. Contents: the 500 operations with oracle answers; all raw model answers (JSONL); scored answers and statistics (Parquet); the executed Colab notebook; analysis, figure and table scripts, including the constant-guess and permutation references, self-consistency checks, OSO consequence, counterfactual hybrid (model plus deterministic code) designs and logistic models with true-SAIL effects; vector figures and LaTeX tables. JARUS documents and model weights are not redistributed.

提供机构:
Zenodo
创建时间:
2026-10-31
二维码
社区交流群
二维码
科研交流群
商业服务