RISE Humanities Data Benchmark Dataset: Complete Stored Results
收藏资源简介:
The RISE Humanities Data Benchmark, maintained by RISE (Research and Infrastructure Support) at the University of Basel, evaluates large language models on tasks from humanities research: transcribing, interpreting and extracting structured data from historical documents such as letters, library cards, newspaper advertisements and medieval manuscripts, scored against ground truths defined by domain experts, with the cost and runtime of every request. Every stored result of the Humanities Data Benchmark as joinable tables: 3,805 runs, 166,380 requests and 3,902,043 metric observations covering data through 2026-10-29. Apart from the framework's own test benchmarks, nothing is filtered: unscored, failed and unresolved runs are rows carrying a status rather than omissions. Tables in Parquet, a metric dictionary, coverage, diagnostics and a source manifest, with the original request and scoring JSON as gzipped JSON Lines. A new release follows each release of the benchmark software; each is a complete snapshot. README.md documents the tables and how to use them, CHANGELOG.md what changed between releases.



