Gnonymous/Web-CogBench

---
license: apache-2.0
task_categories:
- question-answering
language:
- en
- zh
size_categories:
- 1K<n<10K
configs:
- config_name: Exploring_Popup_Close
data_files:
- split: test
path: Exploring/Popup_Close.json
- config_name: Exploring_Single_Step_Exploration
data_files:
- split: test
path: Exploring/Single_Step_Exploration.json
- config_name: Exploring_User_Intent_Prediction
data_files:
- split: test
path: Exploring/User_Intent_Prediction.json
- config_name: Memorizing_Element_Attribute
data_files:
- split: test
path: Memorizing/Element_Attribute_249.json
- config_name: Memorizing_Next_Page_Prediction
data_files:
- split: test
path: Memorizing/Next_Page_Prediction_100.json
- config_name: Memorizing_Source_Element_Prediction
data_files:
- split: test
path: Memorizing/Source_Element_Prediction.json
- config_name: Understanding_Element_Understanding
data_files:
- split: test
path: Understanding/Element_Understanding_sampled_200_clean.json
- config_name: Understanding_WebPage_Understanding
data_files:
- split: test
path: Understanding/WebPage_Understanding_77.json
- config_name: VisualWebBench_Action_Ground
data_files:
- split: test
path: VisualWebBench/VisualWebBench_Action_Ground_103.json
- config_name: VisualWebBench_Action_Prediction
data_files:
- split: test
path: VisualWebBench/VisualWebBench_Action_Prediction_281.json
- config_name: VisualWebBench_Heading_OCR
data_files:
- split: test
path: VisualWebBench/VisualWebBench_Heading_OCR_46.json
- config_name: VisualWebBench_Element_Ground
data_files:
- split: test
path: VisualWebBench/VisualWebBench_element_ground.json
- config_name: VisualWebBench_Element_OCR
data_files:
- split: test
path: VisualWebBench/VisualWebBench_element_ocr.json
- config_name: VisualWebBench_WebQA
data_files:
- split: test
path: VisualWebBench/VisualWebBench_webqa.json
tags:
- web
- agent
- benchmark
---
This benchmark was the Web-CogBench mentioned in the paper [Web-CogReasoner: Towards Knowledge-Induced Cognitive Reasoning for Web Agents.](https://huggingface.co/papers/2508.01858).

The Web-CogBench is used to evaluate [Web-CogReasoner](https://huggingface.co/Gnonymous/Web-CogReasoner), which achieves 84.4 @ Web-CogBench, 86.3 @ VisualWebBench, 30.2% @ WebVoyager, 17.0% and 10.1% @ Online Multimodal-Mind2Web Cross-Tasks and Cross-Webs.

<h3>Statistics of the Web-CogBench</h3>

<table border="1" cellspacing="0" cellpadding="8">
<thead>
<tr>
<th>Cognition</th>
<th>Task Types</th>
<th>Total Samples</th>
</tr>
</thead>
<tbody>
<tr>
<td>Memorizing</td>
<td>
Element Attribute Recognition<br>
Next Page Prediction<br>
Source Element Prediction
</td>
<td>374</td>
</tr>
<tr>
<td>Understanding</td>
<td>
Element Understanding<br>
WebPage Understanding
</td>
<td>277</td>
</tr>
<tr>
<td>Exploring</td>
<td>
User's Intention Prediction<br>
Popup Close<br>
Single Step Exploration
</td>
<td>225</td>
</tr>
</tbody>
</table>