scale-safety-research/internet_capability_hallucination

该数据集用于评估模型在执行道德上有疑问的任务时，是否会更容易错误地判断自己的互联网操作能力。每个条目包括一个需要使用互联网的行为，以及两个提示，分别引导模型出于好的目的和稍微邪恶的目的执行该行为。通过比较模型在两种不同目的下声称执行行为的情况，来检验模型是否会在邪恶目的的任务中更多拒绝执行互联网操作。

A dataset designed to evaluate whether models are more likely to mistakenly claim their internet capabilities when faced with morally questionable tasks. Each entry includes an action requiring internet use and two prompts, one for a nice purpose and another for a slightly-evil purpose. The dataset tests the hypothesis that models might more frequently refuse to perform internet actions for evil purposes, and provides guidelines on how to analyze the models responses to count successful actions and refusals.