Replication package for: It's the Ecosystem, Not the AI — Empirical Evidence that COCOMO II Estimation Failure Predates AI Coding Assistants
收藏资源简介:
Replication package for "It's the Ecosystem, Not the AI: Empirical Evidence that COCOMO II Estimation Failure Predates AI Coding Assistants" (Yang & Saxena, 2026), submitted to Empirical Software Engineering. Contents- notebooks/ : end-to-end Jupyter analysis notebook that reproduces every table and figure in the paper.- data/raw/ : cached public GitHub API responses (pull requests, commit activity, code frequency, repository metadata) for 47 repositories spanning 2004-Q2 to 2026-Q1, plus an LLM inference pricing history.- data/processed/ : bot-filtered quarterly panel, COCOMO II comparison frame, final analysis panel.- data/results/ : per-table output CSVs (placebo DiD, cohort DiD, Chow structural breaks, Granger causality, OLS coefficients, event-study coefficients). Reproduction requires no API keys when using the cached parquet files. See README.md for a one-command quick-start. Methodology summaryUsing a quarterly panel of 47 GitHub repositories (25 modern-ecosystem, 22 traditional) and a multi-signature behavioural bot filter that removes 35.5% of pull-request activity, the paper documents that COCOMO II overestimates effort by ~46x for TypeScript/Python/Rust repositories versus ~4.85x for C/C++/Java — a 9.5x between-group ratio that is stable across bot-filter specifications. Placebo difference-in-differences tests at pre-AI dates (2020-Q4, 2021-Q2) yield ATTs of 1.80 and 1.21 (both p < 0.01), of comparable magnitude to the post-AI ATT at 2022-Q2 (1.74, p = 0.03), identifying ecosystem mismatch rather than AI amplification as the dominant source of COCOMO II's estimation failure. Licences- Code (notebook): MIT — see LICENSE-CODE.- Data (all files under data/): CC-BY-4.0 — see LICENSE-DATA. The redistributed dataset contains no user-level personal data: the pull-request cache records numeric metadata only (pr_number, created_at, merged_at, additions, deletions, review_comments) and does not include user.login, emails, or other contributor identifiers. Commit activity is weekly aggregate totals only. Consumers of this archive must respect GitHub's Terms of Service when performing further collection from the same sources.



