Dataset of Software Self-evolution with SelfEvolve: an Agentic Architecture for Runtime Code Generation

A collection of 11 curated datasets for evaluating AI agents capable of runtime self-evolution through autonomous code generation, codebase integration, and cross-session
composition. Used in ICSE 2026 paper "Self-evolving Systems: a Runtime Architecture for Autonomous Code Generation".

Dataset Categories:

Integration Tasks (4 datasets): Multi-file codebases (577-783 LOC, 7-20 classes)
• Patient Risk Analyzer (Healthcare) - 752 LOC, 12 files, 20 classes
• Student GPA Calculator (Education) - 783 LOC, 10 files, 10 classes
• Salary Analyzer (HR Analytics) - 616 LOC, 13 files, 12 classes
• Inventory Low Stock Alert (E-commerce) - 577 LOC, 8 files, 7 classes

Compositional Tasks (3 datasets): Cross-session capability building
• Matrix Eigenvalue, Portfolio Risk, IoT Sensor Pipeline

Data Processing Tasks (4 datasets): External data manipulation
• Book Recommender (100 books CSV), Friend Suggester (100 users JSON)
• Movie API (100 movies CSV), Performance Tracker (50 reviews CSV)

Statistics:65 files, ~80 KB compressed, 49 Python classes, 2,685 LOC, 9 domains (Healthcare, Education, HR Analytics, E-commerce, Finance, Linear Algebra, IoT, Media,
Social Networks)

Contents: Each dataset includes problem.json (task specification), Python source files (PEP 8 compliant with type hints), __init__.py (package exports), and data files
(CSV/JSON where applicable).

Evaluation Results: Pass@1: 92.7% (WITH TDD) vs 72.7% (WITHOUT TDD); Avg iterations: 2.2 vs 4.7; Wilcoxon test: W=55, p<0.001, r=0.98

Dependencies: Python 3.8+, minimal (NumPy for 1 dataset)

License:CC BY 4.0