Automatically fixing dependency breaking changes

OpenInference Traces for Automated Dependency Update Experiments

Dataset Description

This repository contains a collection of execution traces captured during experiments on automated dependency updates using Large Language Models (LLMs). The traces were recorded using OpenTelemetry and OpenInference (https://github.com/Arize-ai/openinference), providing detailed insights into the behavior and performance of various LLMs in addressing breaking changes caused by dependency updates in Java projects.

Data Format

The traces are stored in JSON Lines format, persisted via a custom OpenTelemetry exporter. This format ensures easy parsing and analysis of the experimental data.

Trace Content

Throughout the execution of the experiments, multiple spans were recorded with custom attributes to facilitate subsequent statistical analysis. These spans capture various aspects of the LLM interactions, including:

1. Input processing2. LLM query generation3. LLM response parsing4. Code modification attempts5. Compilation and test execution results

Research Context

These traces correspond to the experiments described in the associated study, which investigates the effectiveness of zero-shot prompting and agentic approaches in automating dependency updates. The data provides valuable insights into the performance variations across different LLMs and the influence of various factors on repair success rates.

Data Usage

Researchers and practitioners can utilize this dataset to:

1. Replicate the experimental results presented in the thesis2. Conduct further analysis on LLM behavior in software engineering tasks3. Develop and benchmark new approaches for automated dependency updates4. Investigate the decision-making processes of LLMs in code modification tasks