Replication package for "Evaluation and Testing of LLM-Based Agents in Production: A Systematic Literature Review"

Replication package for the systematic literature review "Evaluation and Testing of LLM-Based Agents in Production: A Systematic Literature Review" (submitted to Information and Software Technology). The review followed Kitchenham–Charters guidelines with PRISMA 2020 reporting. It screened 4,670 records from Scopus, Web of Science, and IEEE Xplore (January 2023 – March 2026) and retained 80 primary studies through dual-reviewed full-text assessment, dual data extraction (Cohen's κ = 0.656, Jaccard J = 0.732), and dual-coded quality assessment, with arbiter resolution at every stage. This package contains all machine-readable artefacts needed to reproduce every figure, table, and statistic in the article: - corpus/ — screened corpus: 80 included and 235 excluded studies, plus two title/abstract audits (a 50-paper non-borderline re-audit and a 50-paper snowball citation audit).- extraction/ — dual data extraction (Reviewer 1, Reviewer 2, and arbiter-final; 80 studies × 27 analytical fields, held in 29 CSV columns that also include paper_id and a writing-support citation note), the 705-row field-level discrepancy log, and the extraction protocol.- risk-of-bias/ — three-domain quality assessment (Reviewer 1, Reviewer 2, final) with discrepancies and protocol.- agreement/ — four inter-reviewer agreement reports (screening, extraction, quality assessment, and a non-borderline title/abstract audit).- reproduce.ipynb — a self-contained notebook that runs 320 automated checks against every quantitative claim in the article (passes 320/320 in about 10 seconds).- methodology.md, prisma-2020-checklist.md, kitchenham-2007-compliance.md, statistical-analyses.md — full methodology and item-by-item PRISMA 2020 and Kitchenham–Charters compliance.- primary-studies.bib — BibTeX for the 80 included studies. To reproduce: install the dependencies (pip install -r requirements.txt) and run reproduce.ipynb top to bottom. Expected output: 320 checks passed, 0 failed. Licensed under CC BY 4.0.