Automatically fixing dependency breaking changes

OpenInference Traces for Automated Dependency Update Experiments

Dataset Description

This repository contains a collection of execution traces captured during experiments on automated dependency updates using Large Language Models (LLMs). The traces were recorded using OpenTelemetry and OpenInference (https://github.com/Arize-ai/openinference), providing detailed insights into the behavior and performance of various LLMs in addressing breaking changes caused by dependency updates in Java projects.

Data Format

The traces are stored in JSON Lines format, persisted via a custom OpenTelemetry exporter. This format ensures easy parsing and analysis of the experimental data.

Trace Content

Throughout the execution of the experiments, multiple spans were recorded with custom attributes to facilitate subsequent statistical analysis. These spans capture various aspects of the LLM interactions, including:

1. Input processing2. LLM query generation3. LLM response parsing4. Code modification attempts5. Compilation and test execution results

Research Context

These traces correspond to the experiments described in the associated study, which investigates the effectiveness of zero-shot prompting and agentic approaches in automating dependency updates. The data provides valuable insights into the performance variations across different LLMs and the influence of various factors on repair success rates.

Data Usage

Researchers and practitioners can utilize this dataset to:

1. Replicate the experimental results presented in the thesis2. Conduct further analysis on LLM behavior in software engineering tasks3. Develop and benchmark new approaches for automated dependency updates4. Investigate the decision-making processes of LLMs in code modification tasks

用于自动化依赖项更新实验的OpenInference跟踪数据集

数据集说明
本数据集收录了基于大语言模型（Large Language Model，LLM）开展自动化依赖项更新实验过程中采集的执行跟踪数据。此类跟踪数据通过OpenTelemetry与OpenInference（https://github.com/Arize-ai/openinference）录制，可用于深入剖析各类大语言模型在处理Java项目中因依赖项更新引发的破坏性变更时的行为表现与性能水平。

数据格式
跟踪数据以JSON Lines格式存储，并通过自定义OpenTelemetry导出器持久化。该格式便于对实验数据进行解析与分析。

跟踪内容
在实验执行全程中，系统记录了多条带有自定义属性的跨度（span），以支撑后续统计分析工作。这些跨度涵盖了大语言模型交互的多个关键环节，包括：1. 输入处理；2. 大语言模型查询生成；3. 大语言模型响应解析；4. 代码修改尝试；5. 编译与测试执行结果。

研究背景
本跟踪数据集对应于相关研究中的实验内容，该研究旨在探究零样本提示与智能体驱动方法在自动化依赖项更新任务中的有效性。本数据集可为分析不同大语言模型的性能差异，以及各类因素对修复成功率的影响提供极具价值的参考依据。

数据用途
研究人员与工程实践者可利用本数据集开展以下工作：1. 复现该相关研究论文中展示的实验结果；2. 针对软件工程任务中的大语言模型行为开展进一步分析；3. 研发自动化依赖项更新的新方法并进行基准测试；4. 探究大语言模型在代码修改任务中的决策过程。