aworld_visualwebarena

数据集文件元信息以及数据文件，请浏览“数据集文件”页面获取。

当前数据集卡片使用的是默认模版，数据集的贡献者未提供更加详细的数据集介绍，但是您可以通过如下GIT Clone命令，或者ModelScope SDK来下载数据集

#### 下载方法
:modelscope-code[]{type="sdk"}
:modelscope-code[]{type="git"}

# Recon-Act: A Self-Evolving Multi-Agent Browser-Use System via Web Reconnaissance, Tool Generation, and Task Execution

## json format

```
{
"0": {
"info": {
"fail_error": "",
"observation_metadata": {
"text": {
"url": <currnet_url>
"obs_nodes_info": <text observation info from browser environment, align with the visualwebarena dataset>,
"tabs": <\tabs info>
},
"image": {
"obs_nodes_info": {
<bounding box coordinate (xywh) align with the text obs_nodes_info>
}
}
},
"intent": <intent>,
"input_images": <input_images>,
"ss_ori": <original screenshot relative path>,
"ss_som": <som screenshot relative path>
},
"action": { # same with visualwebarena dataset
"action_type": <action type number>,
"coords": [
0.0,
0.0
],
"element_role": 0,
"element_name": "",
"text": [],
"page_number": 0,
"url": "",
"nth": 0,
"pw_code": "",
"element_id": "",
"key_comb": "",
"direction": "",
"answer": <answer from stop action>,
"raw_prediction": <raw response of action team>,
"action_info": {
"pred_action_history": "",
"pred_action_description": "",
"pred_action": "",
"pred_action_type": "",
"pred_bbox": "",
"pred_type_value": "",
"pred_click_point": "",
"parse_error_msg": "",
"content_to_memo": ""
}
}
}
}
```

action_type see the codes below:

```
class ActionTypes(IntEnum):
"""Valid action types for browser env."""

NONE = 0
# mouse wheel and keyboard, universal across all action spaces
SCROLL = 1
KEY_PRESS = 2

# low level mouse and keyboard actions
MOUSE_CLICK = 3
KEYBOARD_TYPE = 4
MOUSE_HOVER = 5

# mid level mouse and keyboard actions
CLICK = 6
TYPE = 7
HOVER = 8

# page level actions, universal across all action spaces
PAGE_FOCUS = 9
NEW_TAB = 10
GO_BACK = 11
GO_FORWARD = 12
GOTO_URL = 13
PAGE_CLOSE = 14

# high-leval actions that playwright support
CHECK = 15
SELECT_OPTION = 16

STOP = 17
CLEAR = 18
UPLOAD = 19
WAIT = 20
DRAG = 21
```

数据集文件的元信息与数据文件，请前往「数据集文件」页面获取。
当前数据集卡片采用默认模板制作，数据集贡献者未提供更详尽的数据集介绍，但您可通过下述Git Clone命令或ModelScope软件开发工具包（SDK）下载该数据集。

#### 下载方法
:modelscope-code[]{type="sdk"}
:modelscope-code[]{type="git"}

# Recon-Act：基于网页侦察、工具生成与任务执行的自进化多智能体浏览器交互系统

## 格式说明

{
"0": {
"info": {
"fail_error": "",
"observation_metadata": {
"text": {
"url": <currnet_url>
"obs_nodes_info": <text observation info from browser environment, align with the visualwebarena dataset>,
"tabs": < abs info>
},
"image": {
"obs_nodes_info": {
<bounding box coordinate (xywh) align with the text obs_nodes_info>
}
}
},
"intent": <intent>,
"input_images": <input_images>,
"ss_ori": <original screenshot relative path>,
"ss_som": <som screenshot relative path>
},
"action": { # 与视觉网页竞技场（VisualWebArena）数据集一致
"action_type": <action type number>,
"coords": [
0.0,
0.0
],
"element_role": 0,
"element_name": "",
"text": [],
"page_number": 0,
"url": "",
"nth": 0,
"pw_code": "",
"element_id": "",
"key_comb": "",
"direction": "",
"answer": <answer from stop action>,
"raw_prediction": <raw response of action team>,
"action_info": {
"pred_action_history": "",
"pred_action_description": "",
"pred_action": "",
"pred_action_type": "",
"pred_bbox": "",
"pred_type_value": "",
"pred_click_point": "",
"parse_error_msg": "",
"content_to_memo": ""
}
}
}
}

动作类型详见下述代码：

class ActionTypes(IntEnum):
"""浏览器环境下的有效动作类型。"""

NONE = 0
# 鼠标滚轮与键盘操作，适用于所有动作空间
SCROLL = 1
KEY_PRESS = 2

# 低级鼠标与键盘操作
MOUSE_CLICK = 3
KEYBOARD_TYPE = 4
MOUSE_HOVER = 5

# 中级鼠标与键盘操作
CLICK = 6
TYPE = 7
HOVER = 8

# 页面级操作，适用于所有动作空间
PAGE_FOCUS = 9
NEW_TAB = 10
GO_BACK = 11
GO_FORWARD = 12
GOTO_URL = 13
PAGE_CLOSE = 14

# Playwright支持的高级操作
CHECK = 15
SELECT_OPTION = 16

STOP = 17
CLEAR = 18
UPLOAD = 19
WAIT = 20
DRAG = 21