christykl/cua-harm-recovery

该数据集包含人类偏好判断，用于评估计算机使用代理（CUA）伤害场景中的恢复计划。数据集包含226个独特伤害场景中的1,130个标注计划对。每个计划对由两个恢复计划（计划A和计划B）组成，由人类标注者评估哪个计划能更好地解决造成的伤害。数据集结构包括数据字段（如唯一标识符、任务组ID、问题描述、情境描述、计算机状态描述、计划A和B等）和数据分割（1,130个偏好判断和19,210个详细评分问题）。

This dataset contains human preference judgments for evaluating recovery plans in computer use agent (CUA) harm scenarios. The dataset contains 1,130 annotated plan pairs across 226 unique harm scenarios in computer use contexts. Each pair consists of two recovery plans (Plan A and Plan B) that were evaluated by human annotators to determine which plan better addresses the harm caused. The dataset structure includes data fields (such as unique identifiers, task group IDs, question descriptions, situation descriptions, computer state descriptions, Plan A and B, etc.) and data splits (1,130 preference judgments and 19,210 detailed rubric rating questions).