Agentic LLM traces for Simulink Model Repair

This deposit contains chat logs and experimental artifacts from 192 runs used in the paper “Evaluating Large Language Model Agents for Simulink Model Repair.”

The dataset includes interaction traces from Codex CLI (GPT-5.4, GPT-5.2) and Claude Code (Sonnet-4.6), along with the Simulink models and mutation configurations used in the experiments (created with MATLAB R2025b).

Files

llm_traces.zip: JSONL session logs, organized by LLM model and ablation settings. Each file corresponds to a single independent session.

schema.json: Specification of the chat log file naming convention and structure.

simulink_models.zip: Original Simulink models used as the basis for experiments.

mutated_simulink_models.zip: Variants of the base Simulink models with injected mutations.

mutation_injection.zip: MATLAB functions for injecting mutations into given Simulink models to create mutated variants.

test_oracle.zip: Files for the test oracle used by LLM agents to guide the repair process.

Results.xlsx: Contains mutation metadata, fix status, and associated cost metrics for each LLM agent and ablation setting.

analysis.py: Script for computing mutation fix rates by mutation type, LLM, and model category.

usage_calc.py: Script for computing token usage and elapsed time per session, exporting results to Excel.

README.md: Documentation describing dataset structure and usage.