遇见数据集

freightutils-eval: frontier language models on 300 freight reference questions — alone, with the FreightUtils tools, and with web search

收藏
Zenodo2026-10-28 更新2026-10-29 收录
官方服务:

资源简介:

A reproducible measurement of how often frontier language models answer freight reference questions correctly on their own, with the FreightUtils tools (MCP and REST), and with a provider's web search in place of the tools. 300 questions in six families — ADR Table A values, the ADR 2027 amendments, ADR 1.1.3.6 small-load totals, limited and excepted quantities, the HS 2022 hierarchy, freight arithmetic (chargeable weight, CBM, loading metres), and identifiers (UN/LOCODE, AWB prefixes, ULDs) — each with one expected value, scored by a deterministic string rule: no model grades another model. The archive holds the frozen question set (v0.2, sha256 in questions/v0.2/manifest.json), the harness, the scoring rule, and for every run the per-question answers, the summary figures and the spend. Licensing: the question set under questions/ is CC BY 4.0; the harness and scripts are MIT (see LICENSE).

提供机构:
Zenodo
创建时间:
2026-10-28
二维码
社区交流群
二维码
科研交流群
商业服务