CompWoB-Plus

# CompWoB+ CompWoB+ is our extension of the [CompWoB benchmark](https://openreview.net/forum?id=Y9kAsYIjYc), which evaluates language-model agents on sequential compositions of web-automation tasks. Building on the compositional setting introduced by CompWoB, CompWoB+ adds 60 newly constructed HTML environments, with each environment combining three to eight atomic web tasks. ## Dataset Contents ```text CompWoB-Plus/ ├── README.md ├── tasks.jsonl └── html/ └── Gen-*.html ``` - `html/` contains the 60 fixed benchmark environments. - `tasks.jsonl` provides one metadata record for each HTML environment. - Each composition length from three through eight contains 10 environments. | Number of subtasks | Environments | | ---: | ---: | | 3 | 10 | | 4 | 10 | | 5 | 10 | | 6 | 10 | | 7 | 10 | | 8 | 10 | | **Total** | **60** | ## Metadata Schema Each line of `tasks.jsonl` is a JSON object with the following fields: | Field | Type | Description | | --- | --- | --- | | `task_id` | string | Unique environment identifier and HTML file stem | | `html_file` | string | Relative path to the corresponding HTML environment | | `num_subtasks` | integer | Number of atomic tasks in the composition | | `subtasks` | list of strings | Ordered atomic-task sequence | | `split` | string | Dataset split; currently all fixed environments belong to `test` | Example: ```json { "task_id": "Gen-three_click-button_enter-time_edit-text", "html_file": "html/Gen-three_click-button_enter-time_edit-text.html", "num_subtasks": 3, "subtasks": ["click-button", "enter-time", "edit-text"], "split": "test" } ``` ## Usage These HTML files depend on the MiniWoB+/ComputerGym runtime and are not standalone webpages. For dependency installation, environment integration, the evaluation protocol, and runnable code, refer to the [TTED GitHub repository](GITHUB_REPOSITORY_URL_PLACEHOLDER). ## Citation If you use CompWoB+, please cite our TTED article, *Learning Simple Test-Time Environments for LLM Web Agents*: ```bibtex @misc{li2026learning, title = {Learning Simple Test-Time Environments for LLM Web Agents}, author = {Junxuan Li and Zijun Liu and Ziyi Huang and Peng Li and Yuzhou Liu and Ming Yan and Yang Liu}, year = {2026}, note = {Preprint} } ``` The final BibTeX entry will also be provided in the [TTED GitHub repository](GITHUB_REPOSITORY_URL_PLACEHOLDER). ## Acknowledgements CompWoB+ is an extension of the original [CompWoB benchmark](https://openreview.net/forum?id=Y9kAsYIjYc), introduced in *Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web* by Hiroki Furuta, Yutaka Matsuo, Aleksandra Faust, and Izzeddin Gur. We thank the authors for releasing CompWoB and enabling further research on compositional web-agent evaluation. ## License and Attribution For license terms, upstream MiniWoB+/ComputerGym attribution, and citation information, refer to the [TTED GitHub repository](GITHUB_REPOSITORY_URL_PLACEHOLDER).