Replication package for the study "Simple Code, More Scrutiny: Python Proficiency and Review Effort in AI Agent and Human Pull Requests"

Analysis code and derived data for a study of the Python proficiency of code contributed by AI coding agents, compared with human-authored code from the same repositories and with a pre-ChatGPT human baseline, and of the review effort each pull request received. Proficiency is measured with the Code Proficiency Extraction Tool (CPET), which maps Python constructs to the six CEFR levels A1 (Basic) to C2 (Mastery). The study covers 481 merged AI-agent pull requests (GitHub Copilot, Devin, Cursor), 537 merged current-era human pull requests from the same repositories, and 465 merged human pull requests predating the public release of ChatGPT, which separates authorship from the possibility that recent human code is itself AI-assisted. We measure review effort from the GitHub API, counting only human activity. The package is organized by research question. data/ holds the shared per-pull-request tables; collection/ and extraction/ hold the corpus-building and measurement pipeline; rq1/ to rq4/ hold each research question's scripts or notebook together with the LaTeX tables and figures they produce. This archive contains the analysis code and all derived data. The verbatim third-party source corpora that were measured (some 58,000 Python files from roughly 220 repository owners, under a range of licenses that cannot be relicensed here) and the raw per-repository CPET output are not included; they remain in the GitHub repository listed under Related works. We reproduce everything from the results stage onward from what is included here. We redacted personal email addresses from scraped pull-request descriptions in the data tables. No numeric value was altered.GitHub repository of the study: https://github.com/cragkhit/agent-cefr-journal-study.