leosltl/Android-in-the-Wild

---
license: cc-by-4.0
task_categories:
- image-classification
- visual-question-answering
tags:
- android
- mobile
- ui-automation
- screen-understanding
pretty_name: Android in the Wild (AITW)
size_categories:
- 100M<n<1B
---

# Android in the Wild (AITW)

This is a mirror of Google's **Android in the Wild (AITW)** dataset, re-hosted on Hugging Face for easier community access.

## Original Source

- **Paper:** [Android in the Wild: A Large-Scale Dataset for Android Device Control](https://arxiv.org/abs/2307.10088)
- **Original Repository:** [google-research/google-research/tree/master/android_in_the_wild](https://github.com/google-research/google-research/tree/master/android_in_the_wild)

## Dataset Description

Android in the Wild (AITW) is a large-scale dataset for Android device control. It contains human demonstrations of natural language instructions being carried out on Android devices. Each demonstration consists of a sequence of screenshots paired with corresponding actions (taps, swipes, types, etc.) and UI annotations.

## Dataset Structure

The dataset is organized into 5 subsets, stored as **gzip-compressed TFRecord** files:

| Subset | Shards | Description |
|--------|--------|-------------|
| `general` | 321 | General Android tasks |
| `google_apps` | 8,688 | Tasks on Google applications |
| `install` | 1,052 | App installation tasks |
| `single` | 252 | Single-step tasks |
| `web_shopping` | 1,025 | Web shopping tasks |

Additionally, the `splits/` directory contains JSON files defining train/test splits:
- `standard.json`
- `unseen_android_version.json`
- `unseen_domain.json`
- `unseen_subject.json`
- `unseen_verb.json`

## Data Format

Each TFRecord contains examples with the following fields:
- `image/encoded` — screenshot image (encoded)
- `image/ui_annotations_ui_types` — UI element type annotations (e.g., `ICON_STOP`, `ICON_V_BACKWARD`)
- Additional action and metadata fields

## Usage

```python
import tensorflow as tf
import gzip

def read_tfrecord(file_path):
with gzip.open(file_path, 'rb') as f:
raw = f.read()
dataset = tf.data.TFRecordDataset([file_path], compression_type='GZIP')
return dataset
```

For detailed usage instructions, refer to the [original repository](https://github.com/google-research/google-research/tree/master/android_in_the_wild).

## License

This dataset is licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), following the original release.

## Citation

```bibtex
@article{rawles2023android,
title={Android in the Wild: A Large-Scale Dataset for Android Device Control},
author={Rawles, Christopher and Li, Alice and Rodriguez, Daniel and Ber, Oriana and Zitkovich, Brianna},
journal={arXiv preprint arXiv:2307.10088},
year={2023}
}
```

## Disclaimer

This is an unofficial mirror. All credit goes to the original authors at Google Research. This copy is provided solely to facilitate easier access for the research community.

许可证：CC BY 4.0
任务类别：
- 图像分类
- 视觉问答
标签：
- 安卓（Android）
- 移动设备
- UI自动化
- 屏幕理解
美观名称：野外安卓（Android in the Wild, AITW）
数据规模：1亿至10亿样本

# 野外安卓（Android in the Wild, AITW）

本仓库为谷歌**野外安卓（Android in the Wild, AITW）**数据集的官方镜像，已重新托管至Hugging Face平台，以方便全球科研社区便捷获取。

## 原始来源

- **学术论文**：[《野外安卓：面向安卓设备控制的大规模数据集》](https://arxiv.org/abs/2307.10088)
- **原始代码仓库**：[google-research/google-research/tree/master/android_in_the_wild](https://github.com/google-research/google-research/tree/master/android_in_the_wild)

## 数据集概述

野外安卓（AITW）是专为安卓设备控制任务打造的大规模数据集，收录了人类用户在安卓设备上执行自然语言指令的操作演示数据。每条演示数据包含一系列截图、与之对应的操作（点击、滑动、文本输入等）以及UI标注信息。

## 数据集结构

本数据集共分为5个子集，以**gzip压缩TFRecord**格式存储：

| 子集名称 | 分片数量 | 任务描述 |
|--------|--------|-------------|
| `general` | 321 | 通用安卓任务 |
| `google_apps` | 8688 | 谷歌官方应用相关任务 |
| `install` | 1052 | 应用安装任务 |
| `single` | 252 | 单步操作任务 |
| `web_shopping` | 1025 | 网络购物任务 |

此外，`splits/` 目录下包含用于定义训练集/测试集划分的JSON文件：
- `standard.json`
- `unseen_android_version.json`
- `unseen_domain.json`
- `unseen_subject.json`
- `unseen_verb.json`

## 数据格式

每个TFRecord文件包含以下字段的示例数据：
- `image/encoded` — 编码后的截图图像
- `image/ui_annotations_ui_types` — UI元素类型标注（例如 `ICON_STOP`、`ICON_V_BACKWARD`）
- 额外的操作与元数据字段

## 使用方法

python
import tensorflow as tf
import gzip

def read_tfrecord(file_path):
with gzip.open(file_path, 'rb') as f:
raw = f.read()
dataset = tf.data.TFRecordDataset([file_path], compression_type='GZIP')
return dataset

如需获取详细使用指南，请参阅[原始代码仓库](https://github.com/google-research/google-research/tree/master/android_in_the_wild)。

## 许可证

本数据集遵循原始发布协议，采用[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)知识共享署名许可协议进行授权。

## 引用格式

bibtex
@article{rawles2023android,
title={Android in the Wild: A Large-Scale Dataset for Android Device Control},
author={Rawles, Christopher and Li, Alice and Rodriguez, Daniel and Ber, Oriana and Zitkovich, Brianna},
journal={arXiv preprint arXiv:2307.10088},
year={2023}
}

## 免责声明

本仓库为非官方镜像，所有学术荣誉归属于谷歌研究院的原作者团队。本副本仅为方便科研社区便捷获取而发布。