orlando23/screendata

Android Screen Hierarchical Layout dataset (ASHL) 用于训练层次布局树的检测模型。该数据集包含来自2,180个屏幕截图的约52,000个边界框，这些边界框被标记为全局或局部区域，涵盖了18个不同的Android应用程序。数据集的收集过程基于MagicWand模拟器完全自动化，首先根据预记录的人类导航不同Android应用程序的操作来采样屏幕截图及其对应的源代码，然后从源代码中提取具有预定义坐标的GUI区域$R_i$，这些区域成为树结构中的节点和叶子。

The Android Screen Hierarchical Layout dataset (ASHL) is used to train a detection model for the Hierarchical Layout Tree. This dataset contains around 52,000 bounding boxes from 2,180 screenshots, covering 18 different Android applications. The bounding boxes are labeled as either global or local regions. The collection process of the dataset is fully automated, based on the MagicWand simulator, where screenshots and their corresponding source codes are sampled based on pre-recorded actions of humans navigating different Android applications. Then, GUI regions $R_i$ are extracted from the source codes with predefined coordinates to become nodes and leaves in a tree structure.

原始信息汇总

Android Screen Hierarchical Layout dataset

概述

• 名称: Android Screen Hierarchical Layout dataset (ASHL)
• 用途: 用于训练层次布局树的检测模型
• 数据量:
  • 约52,000个边界框
  • 2,180张截图
  • 涵盖18个不同的Android应用程序
• 标签: 边界框标记为全局或局部区域
• 数据收集:
  • 基于MagicWand模拟器自动收集
  • 通过预先记录的人类导航不同Android应用程序的动作进行截图和源代码采样
  • 从源代码中提取GUI区域，形成树结构的节点和叶子

引用

• BibTeX: bibtex @misc{fan2024readpointedlayoutawaregui, title={Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding}, author={Yue Fan and Lei Ding and Ching-Chen Kuo and Shan Jiang and Yang Zhao and Xinze Guan and Jie Yang and Yi Zhang and Xin Eric Wang}, year={2024}, eprint={2406.19263}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2406.19263}, }

  @inproceedings{dingenhancing, title={Enhancing Mobile" How-to" Queries with Automated Search Results Verification and Reranking}, author={Ding, Lei and Zhang, Yi and Bheemanpally, Jeshwanth}, booktitle={The Second Workshop on Generative Information Retrieval} }