遇见数据集

Study Materials, Human Coding, and Reproduction Files for Preserving Requirements Across AI Updates

收藏
Zenodo2026-11-02 更新2026-11-05 收录
官方服务:

资源简介:

Study materials, human coding records, and numerical reproduction files accompanying the study "Preserving Requirements Across AI Updates: Task Notes, Human Review, and Decisions After Interruption".Version 1.1.0 retains the complete human-coding archive from version 1.0.0 and adds Supplement S1 (PDF and matching Markdown source), frozen generation inputs and Chinese prompts, request configuration, questionnaire wording, executable analysis code, deidentified numerical and coded-response data, and archived statistical output. Start with 00_START_HERE_EN.txt. The methods ZIP contains its own README and file index. The original README_EN.txt and SHA256SUMS.txt describe the human-coding archive after the metadata-only correction; SHA256SUMS_v1.1.0.txt covers the additional release files.The readable supplement documents study sequence and sample definitions, coding boundaries, initial independent reliability and adjudication history, complete note-state and transition counts, interface tests, observational and sensitivity analyses, the original questionnaire, quotations, and reproduction details. The reproduction script verifies specified counts, test families, models, option frequencies, and field-level reliability against archived output. The META appendix itself belongs in the manuscript PDF.Human coding materials and codebook:The archive contains 340 Excel workbooks: 287 cleaned copies of existing workbooks, 52 workbook conversions of CSV records, and one file index. Materials are organized by pilot coding and rule development, initial human coding, discrepancy review, final adjudication, statistical and coverage summaries, codebook versions, and historical templates. The current codebook and field-option dictionary are also included.The behavioral study comprises 900 sessions. Formal human coding covers 876 records per material category and 23 fields, yielding 20,148 final field-level labels. The three note stages (T1, T2, T3) share the same 876 participants. Each of the six open-ended questions has 876 formal records, but the excluded pilot participants differ across questions. Twenty-four records per category were used for pilot coding and rule development.Within each formal category, 676 records received initial double coding and 200 received single coding. Initial shared batches comprised 76, 200, 175, 175, and 50 records. In the last batch, each coder coded 150 records: 100 unique and 50 shared. The 15,547 valid initial field-level pairs contained 14,138 exact agreements (90.9%). Subsequent discrepancy review and adjudication are separate stages and must not be interpreted as new independent reliability measurements.AI-assisted suggestions were shown during discrepancy review. Of 1,314 initial discrepancies in rounds 2-5, 1,001 agreed after review and 313 remained different. These 313 and 95 additional discrepancies from round 6 were resolved by 408 human final adjudications. One initially missing entry was separately supplemented. The distributed workbooks remove coding-assistance suggestion columns, their accompanying rationales, and automatically generated item-specific review hints; this does not change the history of AI assistance. The study's AI-generated T2 notes and participants' references to AI are retained as research materials. Human choices, comments, and final adjudications were not rewritten or filled using AI suggestions.All 408 final adjudications were checked against the submitted snapshot. Verification compared 1,937,108 retained nonempty cells; all 340 workbooks were reopened and the ZIP passed integrity checks. The original 2,206 workbook formulas preserve their logic, with seven references shifted to reflect deleted columns. Historical blank templates and earlier rule versions remain explicitly separated. Thirty-two known pilot rule-difference groups were preserved without recoding and are not part of the formal final semantic analysis.Author-specific document properties, local machine paths, and platform identity-linkage columns have been removed from this distribution. Original audit sources and private provenance mappings are excluded. The Excel index uses relative paths. Most source materials and the codebook are in Chinese; the metadata and this accompanying README provide an English guide.Minor file correction (November 2, 2026): Residual internal local-path metadata was removed from 55 workbooks. All worksheet content, formulas, styles and remaining workbook members are unchanged. Four quotation-ledger references in Supplement S1 were aligned to Figure 8 in the final manuscript. No participants were added or recoded; no research findings were changed.

提供机构:
Zenodo
创建时间:
2026-11-02
二维码
社区交流群
二维码
科研交流群
商业服务