autoprogrammer/cs527-optical-compression-trajectories

---
license: mit
task_categories:
- text-generation
language:
- en
tags:
- code
- swe-bench
- optical-compression
- agent-trajectories
- mini-swe-agent
- gpt-5-mini
pretty_name: "CS527 Optical Compression Agent Trajectories"
size_categories:
- n<1K
---

# CS527 Optical Compression Agent Trajectories

Agent trajectories from the paper **"Optical Compression for Agentic Code Understanding"** (CS 527 Group-9, UIUC).

## Overview

This dataset contains 200 agent trajectories (100 per condition) from evaluating optical compression on SWE-bench Verified using GPT-5-mini and mini-swe-agent.

- **Text condition**: Standard text-based agent (all tool outputs as plain text)
- **Optical condition**: Code-heavy tool outputs rendered as monospace images

Both conditions use the same 100-instance stratified subset of SWE-bench Verified.

## Key Results

| Condition | Resolve Rate | Avg Steps | Avg Cost |
|-----------|-------------|-----------|----------|
| Text | 51/100 (51%) | 21.3 | $0.03 |
| Optical | 51/100 (51%) | 19.8 | $0.06 |

## Structure

```
text/ # Text condition (100 instances)
├── preds.json # Predictions
├── <instance_id>/
│ └── <instance_id>.traj.json # Full trajectory
optical/ # Optical condition (100 instances)
├── preds.json
├── <instance_id>/
│ └── <instance_id>.traj.json
```

## Trajectory Format

Each `.traj.json` file follows the mini-swe-agent trajectory format (`mini-swe-agent-1.1`) and contains:

- `messages`: Full conversation history (system, user, assistant, tool messages)
- `info.model_stats`: Token usage and cost
- `info.exit_status`: How the agent terminated (Submitted / LimitsExceeded)
- `info.submission`: The generated patch

For optical trajectories, additional fields include:
- `info.observations_rendered`: Number of tool outputs rendered as images
- `info.observations_total`: Total number of tool outputs
- `info.total_images`: Total images generated
- `info.wall_clock_time`: End-to-end time in seconds

## Dataset Details

- **Model**: GPT-5-mini (gpt-5-mini-2025-08-07)
- **Framework**: [mini-swe-agent](https://github.com/SWE-agent/mini-swe-agent)
- **Benchmark**: SWE-bench Verified (100-instance stratified subset)
- **Rendering**: 12pt DejaVu Sans Mono, 120-char width, 80 lines/image

## Links

- **Paper & Code**: [github.com/Rachum-thu/cs527-proj](https://github.com/Rachum-thu/cs527-proj)
- **SWE-bench**: [princeton-nlp/SWE-bench_Verified](https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified)

## Citation

```bibtex
@misc{tian2026optical,
title={Optical Compression for Agentic Code Understanding},
author={Tian, Runchu and Reddy, Vikas},
year={2026},
note={CS 527 Course Project, UIUC}
}
```