遇见数据集

Paper 7, Study 1: locked run records

收藏
Zenodo2026-10-21 更新2026-11-05 收录
官方服务:

资源简介:

The locked run records of the first study of the paper More Sessions Choose a Temporary Fallback After a Replayed Agent Memory Summary Than After the Run Log (doi:10.5281/zenodo.23427117), published after the confirmatory run was locked and analysed; the study's registration did not provide for their publication, which is the author's later decision and changes no registered number. The study's protocol, code and stimuli were frozen, deposited and timestamped before its first smoke or confirmatory call (doi:10.5281/zenodo.23291437). The study replays, with declared departures, the memory pipeline of OpenAI's open-source Codex agent at a pinned commit (default v1 and opt-in v2 instructions) over scripted runs of a recurring task: eight successes of a faster standard route, then one unexplained failure or a ninth success, and further scripted runs; ReasoningBank's released prompts and two harness records (a dated log; per-route counts) write memory from the same runs. Sessions of eight reader models, shown the memory, forecast that the standard route would succeed and name the route they would use; no choice is scored. 96 streams in 16 domains. The record holds the confirmatory run (9,600 writer, 45,216 reader and 2,304 coder requests; 57,131 files) and the smoke (1,190 requests) exactly as locked, each with its completion manifest (confirmatory SHA-256 05d18280d05017fb39fa43ba3b8adebf5638f5b0ee0996d879c7a508489e9476) and its OpenTimestamps proof as stamped, with the confirmatory proof's Bitcoin attestation (block 971004; the smoke's proof was not upgraded); the freeze manifest's attestation; the registered analysis's output and its report tables; the independent re-implementation of the twelve registered contrasts with its comparison; the run log and the launcher logs (with local paths replaced and two passages on another of the author's works withheld, as the README lists); and a verification script. Each record holds a request's identity, the prompt's hashes, the typed status and answer, the attempts, the cost and, for each attempt that has one, the provider's raw response; the raw responses of OpenAI's API echo the request's system prompt. README.md describes every file and names, with counts, the texts that the providers and software libraries wrote (refusal notices, error and billing diagnostics); SHA256SUMS lists every file, the files of the folders reports/, reimpl/ and launcher_logs/ among them, and the zip archives of the same names in which those folders are uploaded. The records and logs also show the runs' times, recorded costs and spending caps, whether each provider's API key was set (never a key), the providers' response identifiers and the software used, as the README lists. The paper's second version (doi:10.5281/zenodo.23524245, published on 2026-10-20) predates this deposit and still says that these records are not public. The tasks are synthetic: every task, route, tool, path, date and earlier run in the scripted histories that the models read was written by the study's scripts (the writers' instructions are those of the replayed memory pipelines, with their examples), and what the models wrote about them is model output, kept as received, not a statement of fact by the author. Licences (mixed; LICENSE.md): code MIT; data and documents written for the study CC BY 4.0 (the re-implementation's files, which a language model wrote for the study, under the same terms by kind); text written by the models, by the providers (refusal notices, error and billing diagnostics, the responses' fields) and by software libraries is reproduced as received and not licensed here; Codex's text that the responses echo stays under the Apache License 2.0 (CODEX_LICENSE, CODEX_NOTICE).

提供机构:
Zenodo
创建时间:
2026-10-21
二维码
社区交流群
二维码
科研交流群
商业服务