遇见数据集

Pre-call feasibility artefact for "Before the First Model Call: Certifying the Constructibility of LLM-Based Software Engineering Benchmarks"

收藏
Zenodo2026-08-04 更新2026-08-13 收录
官方服务:

资源简介:

Research artefact for the Short Communication "Before the First Model Call:Certifying the Constructibility of LLM-Based Software Engineering Benchmarks",prepared for Information and Software Technology. No model calls were made inthe work this artefact records. It holds no model output and no result rows. closeout_v9 is the sealed package: 20 payload files and its own SHA-256manifest, byte-identical to the seal of 3 August 2026. It holds the capacityreport with the raw probe logs, the block-allocation certificate, theindependence ledger with alias detection, the 47-item batch with its fourvalidation layers, and the correction records with the identities before andafter each correction. reproduction holds 15 scripts and outputs, among them the allocation solver,the Table 1 generator, and the retrospective validation. Six of the sevenretrospective rows replay from the deposit alone. The seventh, R3, needs theoriginal frozen container and provenance environment, which are notredistributed; its sealed result is included and labelled as such rather thanpresented as a fresh run. To verify: shasum -a 256 -c SHA256SUMS On the capacity figure. The 86 admissible items are conditional on the frozenenvironment, endpoint, admissibility rules and probe catalogue. 24 of the 86rest on executed probes; the remaining 62 rest on documented per-classjudgements, each resolved in favour of the design. The figure is an estimate ofwhat this catalogue supplies, not a bound on what the toolchains can produce.

提供机构:
Zenodo
创建时间:
2026-08-04
二维码
社区交流群
二维码
科研交流群
商业服务