遇见数据集

Preregistration and materials for a 2x2 experiment on conceptual armament and the rated originality of generated research ideas

收藏
Zenodo2026-11-03 更新2026-11-05 收录
官方服务:

资源简介:

Version 2 adds the preregistration and materials of a follow-up minimal experiment with strong restraint, conducted on 2 and 3 November 2026. The outputs of conditions 1 and 3 of the original experiment (no armament and ten concepts, both without search) were reused, and two new conditions added the instruction "Base your answer only on what is supported by established research, and avoid unverified inference" (condition 5 without armament and condition 6 with the ten concepts). The same generator produced each new condition once in a new session, and the same two judges, blind to conditions, scored the forty pooled predictions for originality and internal coherence. The archive CAE-minimal-restraint-experiment_2026-11-03.zip holds two parallel sets, ja/ and en/, each with a reading edition that gathers the files into a single document. The Japanese set is authoritative, and the English set is a translation. It includes the instructions and the predictions registered before execution, the exact text pasted for condition 6, the raw outputs of conditions 5 and 6, the blinded judging input with its key, the full text handed to the judges, both judges' raw scores, the aggregation by condition, the count of indicator 8a and a timeline. Two records of error are included deliberately. Two runs of condition 6 in which the ten concepts did not reach the generator are retained and not scored. Indicator 8a, which was part of the registered predictions, was counted only after the originality results were known; the passages counted are listed verbatim. The description of version 1 follows. This deposit contains the preregistration and the complete materials for a 2x2 experiment on conceptual armament and the rated originality of generated research ideas. The experiment held one generator constant and varied whether it was armed with ten concepts published after the recorded training cutoffs of the models used, and whether it could search the live web. Two judges, blind to conditions, scored the pooled output. The deposit includes the preregistration with its four falsification conditions and the registered per-condition predictions, the protocol, the four condition prompts, the concept payload, the scoring rubric for the eight indicators, the raw generation outputs for all four conditions including one deviated run that was repeated and excluded from analysis, the blinded scoring input with its two keys, the corrected aggregation, the recoverability round protocol and its delivered file, the external citation verification record, and the item-level decomposition. The experiment was conducted in Japanese. The archive holds two parallel sets of the nineteen files, ja/ and en/, together with a reading edition of each that gathers the same nineteen files into a single document. The Japanese set is authoritative: every instruction was administered in Japanese, every generated output is Japanese, and all scoring was performed on the Japanese text. The English set is a translation, and each English file names the Japanese file it corresponds to. Where a file reproduces material that was actually handed to a model or to a judge, the Japanese original is given alongside the translation. Two records of error are included deliberately. One aggregation used the wrong key and was rebuilt; the superseded files are retained with an invalidation notice. One generation run touched a repository status file before generating and was repeated; the deviated run is retained and is not scored.

提供机构:
Zenodo
创建时间:
2026-11-03
二维码
社区交流群
二维码
科研交流群
商业服务