Agentic LLM traces for Simulink Model Repair

This deposit contains chat logs and experimental artifacts from 192 runs used in the paper “Evaluating Large Language Model Agents for Simulink Model Repair.”

The dataset includes interaction traces from Codex CLI (GPT-5.4, GPT-5.2) and Claude Code (Sonnet-4.6), along with the Simulink models and mutation configurations used in the experiments (created with MATLAB R2025b).

Files

llm_traces.zip: JSONL session logs, organized by LLM model and ablation settings. Each file corresponds to a single independent session.

schema.json: Specification of the chat log file naming convention and structure.

simulink_models.zip: Original Simulink models used as the basis for experiments.

mutated_simulink_models.zip: Variants of the base Simulink models with injected mutations.

mutation_injection.zip: MATLAB functions for injecting mutations into given Simulink models to create mutated variants.

test_oracle.zip: Files for the test oracle used by LLM agents to guide the repair process.

Results.xlsx: Contains mutation metadata, fix status, and associated cost metrics for each LLM agent and ablation setting.

analysis.py: Script for computing mutation fix rates by mutation type, LLM, and model category.

usage_calc.py: Script for computing token usage and elapsed time per session, exporting results to Excel.

README.md: Documentation describing dataset structure and usage.

本数据集包含论文《面向Simulink模型修复的大语言模型智能体评估》（Evaluating Large Language Model Agents for Simulink Model Repair）中192次实验所用的聊天日志与实验产出物。

本数据集涵盖Codex CLI（GPT-5.4、GPT-5.2）与Claude Code（Sonnet-4.6）的交互轨迹，同时包含实验所用的Simulink模型与变异配置（实验基于MATLAB R2025b构建）。

文件
llm_traces.zip：采用JSONL格式的会话日志，按大语言模型与消融实验设置进行组织，每个文件对应一次独立会话。
schema.json：聊天日志文件的命名规范与结构说明文档。
simulink_models.zip：作为实验基础的原始Simulink模型。
mutated_simulink_models.zip：注入变异后的基础Simulink模型变体。
mutation_injection.zip：用于向指定Simulink模型中注入变异以生成变体的MATLAB函数集。
test_oracle.zip：供大语言模型智能体用于指导修复流程的测试预言相关文件。
Results.xlsx：包含各大语言模型智能体与消融实验设置的变异元数据、修复状态及相关成本指标。
analysis.py：用于按变异类型、大语言模型及模型类别计算变异修复率的脚本。
usage_calc.py：用于计算每次会话的Token使用量与耗时，并将结果导出至Excel的脚本。
README.md：说明数据集结构与使用方法的文档。