el7982/aware-bench

AWARE-bench是一个用于评估语言模型中评估意识的基准数据集，即衡量模型在被评估时与在实际部署时的行为差异程度。该数据集作为AWARE-bench的配套数据，包含三个主要文件夹：collected_trajectories/存储按模型划分的原始轨迹JSON文件、部署聊天记录和聊天评估提示；measure_logs/包含测量阶段输出，如评估日志和导出的CSV分数文件；paper_replication/提供用于生成论文图表和ANOVA表格的已评分CSV数据。数据集支持表格分类和问答任务，标签涉及评估意识、LLM基准、轨迹分析、探测和消融研究等，旨在帮助研究者分析和理解语言模型在评估环境下的行为变化。Companion dataset for AWARE-bench, a benchmark for evaluation awareness in language models — the degree to which a model behaves differently when it perceives it is being evaluated versus deployed. The dataset includes three folders: collected_trajectories/ for raw trajectory JSONs per model, deployment chats, and chat-eval prompts; measure_logs/ for measure-stage outputs such as .eval logs and CSV exports of scores; and paper_replication/ for scored CSVs that feed the paper figures and ANOVA tables. It supports tabular-classification and question-answering tasks, with tags like evaluation-awareness, llm-benchmark, trajectories, probing, and ablations, facilitating research on model behavior under evaluation conditions.