Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation

Replication Package: Software Self-Extension with SelfEvolve Submitted to: SEAMS 2026 (International Symposium on Software Engineering for Adaptive and Self-Managing Systems) Paper Title: Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation Overview This replication package contains datasets, empirical results, and supplementary materials for evaluating SelfEvolve—an architecture enabling runtime self-extension through autonomous code generation, codebase integration, and cross-session composition. Complete implementation and baseline comparison code: https://anonymous.4open.science/r/self-evolving-systems-FEC4 Contents 1. Evaluation Datasets (11 Tasks) Curated benchmark for evaluating runtime codebase integration capabilities across three task categories: Integration Tasks (4): Multi-file codebase navigation and integration Patient Risk Analyzer (Healthcare): 752 LOC, 12 files, 20 classes Student GPA Calculator (Education): 783 LOC, 10 files, 10 classes Salary Analyzer (HR Analytics): 616 LOC, 13 files, 12 classes Inventory Low Stock Alert (E-commerce): 577 LOC, 8 files, 7 classes Compositional Tasks (3): Cross-session capability building Matrix Eigenvalue Composition (Linear Algebra) Portfolio Risk Assessment (Finance) IoT Sensor Pipeline (IoT Systems) Data Processing Tasks (4): External dataset manipulation Book Recommender (100 books, CSV) Friend Suggester (100 users, JSON) Movie API (100 movies, CSV) Performance Tracker (50 reviews, CSV) 2. Empirical Results TDD Pipeline Ablation Study: With TDD: 51/55 runs (92.7%), avg 2.2 iterations Without TDD: 40/55 runs (72.7%), avg 4.7 iterations Statistical significance: Wilcoxon signed-rank test W=55, p<0.001, r=0.98 Baseline Framework Comparison (165 experimental runs): SelfEvolve: 51/55 (92.7%) AutoGen: 17/55 (30.9%) MetaGPT: 15/55 (27.3%) AgentCoder: 0/55 (0.0%) Statistical significance: Wilcoxon + Bonferroni correction (α=0.0167), all p_adj < 0.012 3. Supplementary Materials Appendix: Qualitative architectural comparison: Evidence for 11 capabilities with direct quotations from papers Quantitative empirical evaluation: Complete methodology, failure analysis, statistical tests Complete Codebase (anonymous.4open.science): SelfEvolve implementation with all components Baseline configurations for fair comparison All 165 evaluation results with detailed logs Statistical analysis scripts Technical Specifications Requirements: Python 3.10+, minimal dependencies Statistical Methodology: Problem-level pairing (N=11 paired observations) Wilcoxon signed-rank test (non-parametric) Bonferroni correction for multiple comparisons (α=0.05/3=0.0167) License Datasets, Code, and Appendix: CC BY 4.0 Reproducibility: All results can be reproduced using provided datasets and codebase. See documentation in repository. Keywords: Self-evolution, Runtime Code Generation, LLM Agents, Multi-Agent Systems, Test-Driven Development, Codebase Integration, Cross-Session Composition