cua-lite/OpenCUA

---
license: other
tags:
- cua-lite
- gui
- sft
task_categories:
- image-text-to-text
configs:
- config_name: default
data_files:
- split: train
path:
- "*/*/train*parquet"
- "*/*/train/*.parquet"
- "*/*/train/*/*.parquet"
- split: validation
path:
- "*/*/validation*parquet"
- "*/*/validation/*.parquet"
- "*/*/validation/*/*.parquet"
- config_name: desktop.navigation
data_files:
- split: train
path:
- "desktop/navigation/train*parquet"
- "desktop/navigation/train/*.parquet"
- "desktop/navigation/train/*/*.parquet"
- split: validation
path:
- "desktop/navigation/validation*parquet"
- "desktop/navigation/validation/*.parquet"
- "desktop/navigation/validation/*/*.parquet"
---

# cua-lite/OpenCUA

cua-lite preprocessed version of xlangai/AgentNet (OpenCUA). Desktop navigation trajectories from Ubuntu and Windows/Mac environments with pyautogui-style actions converted to CUA-lite tool calls.

## Origin

- [https://huggingface.co/datasets/xlangai/AgentNet](https://huggingface.co/datasets/xlangai/AgentNet)

## Load via `datasets`

```python
from datasets import load_dataset

# entire dataset
ds = load_dataset("cua-lite/OpenCUA")

# just one (platform, task_type) cohort
ds = load_dataset("cua-lite/OpenCUA", "desktop.navigation")
```

You can also filter by `metadata.platform` / `metadata.task_type` /
`metadata.others.*` after loading; every row carries a rich `metadata`
struct (see schema below).

## Schema

Each row has these columns:

| column | type | notes |
|---|---|---|
| `images` | list[Image] | embedded PNG/JPEG bytes; HF viewer renders thumbnails |
| `messages` | list[struct] | OpenAI-style turns with `role` + structured `content` |
| `metadata` | struct | `{platform, task_type, extra_tools, valid_actions, others{...}}` |

Coordinate values in `messages` are normalized to `[0, 1000]` integers.

## Layout

```
<platform>/<task_type>/<split>/shard-NNNNN-of-NNNNN.parquet # single-variant cohort
<platform>/<task_type>/<split>/<variant>/shard-NNNNN-of-NNNNN.parquet # multi-variant cohort
```

- `platform` ∈ {desktop, mobile, web}
- `task_type` ∈ {understanding, grounding.action, grounding.point, grounding.bbox, navigation} — used verbatim as the dir component
- HF config names are `<platform>.<task_type>` (e.g. `mobile.grounding.action`). The agent registry lookup key in code is `<agent>@<platform>@<task_type>` (e.g. `qwen3_vl@[e-mail]`); only this user-facing token uses `.` between platform and task_type, because `@` triggers a 403 on the dataset-viewer's signed image URLs.
- HF split names stay `train` / `validation` (the `datasets` library blacklists `<>:/\|?*` in split names; everything else is fine in config_name)
- `validation` is an in-distribution held-out slice (never used in training); `test` is reserved for out-of-distribution benchmark datasets

## Stats

| platform | task_type | variant | train | validation |
|---|---|---|---:|---:|
| desktop | navigation | ubuntu | 4,890 | 102 |
| desktop | navigation | win_mac | 17,173 | 371 |

## Local mirror & SFT export

For local workflows (SFT export, dedup, mixing across datasets), use
`lite.data.hf.download` to mirror this repo back to the canonical local
layout:

```
$CUA_LITE_DATASETS_ROOT/cua-lite/OpenCUA/
images/<hash[:2]>/<hash>.<ext> # content-addressed image store
<platform>/<task_type>/<split>[/<variant>].parquet # rows reference images by relative path
```

Rows in the local parquet have `images: list[str]`; bytes are extracted to
the image store. `lite.train.utils.data.export_sft` consumes the local
form directly with `--image-root=$CUA_LITE_DATASETS_ROOT`.

- Total unique images: **392,526**
- Image store size: **186.57 GB**

## Notes

_(none)_

## License & citation

See original dataset (xlangai/AgentNet).

See https://huggingface.co/datasets/xlangai/AgentNet