Agent-Mediated Phase Transitions in Software Delivery: Search, Screening and Coding Data

Single-coder extraction, checked against an independent coder on a sample. First-pass screeningand coding were performed by a single coder. The 10% sample (dual_screen_sample = YES, n = 19,drawn with random.Random(7410238)) was screened twice, on two separate checks with differentepistemic value: Intra-coder test–retest — the same coder, in a fresh session and blind to its own first decision, reproduced 18 of 19 decisions: 94.7% agreement, Cohen's kappa = 0.883. This measures stability under repetition and cannot detect a bias shared by both passes. Inter-coder agreement — a second, independent coder (the author), blinded to the first pass, agreed on 14 of 19: 73.7% agreement, Cohen's kappa = 0.463 (moderate; SE 0.206; 95% CI 0.06–0.87). Five records were decided differently: A024, A034, A052 and A070 (first coder excluded, second coder admitted) and A150 (first coder admitted, second coder excluded). The second coder's decisions were adopted, so the included set is 65 of 194 in-window records; the four admitted records were coded on all eight facets. Both statistics are reported separately in 08_dual_screening.json, which marks inter_rater_agreement as the primary statistic and sets WARNING to null. 08_human_coder_decisions.csv records the second coder's decisions together with the first-pass decision and rule for each sampled record, so the statistics stay reproducible now that the adopted decisions sit in the corpus. The 19 sampled records carry a decision agreed by two coders; the remaining 175 carry the first coder's decision alone. The corpus is therefore screened to two standards, an asymmetry the manuscript states in Section 3.8 and carries into Section 6.2 as a threat to validity. Grey literature only, in this strand. arXiv is coded as grey literature and is never pooled withpeer-reviewed evidence. The eight peer-reviewed databases in Table 4 are pending institutional access; theirresults must be appended before the frequency tables are treated as the study's final result. Screening aid, not decision. 02_screening_sheet.csv is a keyword triage aid. The decisions in03_screening_decisions.csv were made on the abstract text by the coder; the aid's proposed bucket is retained only for transparency about how attention was allocated. Time window. Four records pre-date January 2023 and are excluded by the registered window. Terminology instability. The constructive threat registered in the protocol (Section VII) is observable in thedata: 04_analysis_RQ4_seed_crosswalk.csv shows Section V phenomena appearing under unrelated vocabulary, which is why single-term booleans under-recall this literature. v4 adopts the second coder's decisions into the corpus. The 10% sample was screened by the author as second coder; agreement with the first-pass decisions was 14 of 19 records (73.7%, Cohen's kappa = 0.463, moderate), against 18 of 19 (94.7%, kappa = 0.883) for the first coder's own blinded re-screen. The four records the second coder admitted (A024, A034, A052, A070) were coded on all eight facets and A150 was removed, so the corpus is now 65 included studies of 194; every count and table was recomputed. 08_human_coder_decisions.csv now also carries the first-pass decision and rule for each sampled record, so the reliability statistics stay reproducible after adoption. The manuscript reports both figures and treats their divergence as the study's principal limitation.