遇见数据集

SPECFORM-H1 and SPECFORM-H2: Renderer-format theory-to-program translation datasets (v1.0.0)

收藏
Zenodo2026-08-10 更新2026-08-13 收录
官方服务:

资源简介:

SPECFORM-H1 and SPECFORM-H2 are companion datasets from a prospective, preregistered, randomized experiment testing whether the presentation format of a verbal theory changes the executable program produced from it by a large language model. The experiment compared a structured intervention-contract representation with connected prose while controlling the theory content, randomized assignment, model snapshots, and evaluation procedure. The archive contains a shared generation corpus of 320 compiled and validated programs, together with two distinct experimental stages: SPECFORM-H1 evaluates single-input challenges using central finite differences. SPECFORM-H2 evaluates interaction challenges using mixed second finite differences. Both stages execute the same 320 programs but use different probe batteries and estimands. They are separate hypotheses and should not be pooled or interpreted as independent samples. The deposit includes parsed and compiled programs, generation and execution records, receipts, provenance information, forward and reverse response-tensor archives, execution summaries, analysis artifacts, schemas, export documentation, and a complete SHA-256 manifest. Account-scoped raw provider objects and sensitive transport metadata are excluded, while cryptographic digests preserve the connection between the released derivatives and the retained source evidence. The registered scientific verdict for H1, H2, and their conjunction is NOT_SUPPORTED: the experiment did not establish the full preregistered claim that renderer format reliably changes theory-to-program translation under all registered endpoints, transforms, and controls. The complete null result and retained secondary structure make the corpus suitable for research on autoformalization, executable theory construction, representational sensitivity, finite-difference response geometry, reproducible LLM experimentation, and alternative prospectively specified analyses. Companion reproduction software: https://doi.org/10.5281/zenodo.21879576 Data are released under the Creative Commons Attribution 4.0 International licence.

提供机构:
Zenodo
创建时间:
2026-08-10
二维码
社区交流群
二维码
科研交流群
商业服务