SkillFactory/BF_EVAL-cd3args-Qwen2.5-1.5B-Instruct-R1-SFT

---
license: mit
task_categories:
- text-generation
language:
- en
tags:
- evaluation
- skill-factory
---

These datasets are exactly like the Evaluation datasets except the model_responses array are budget forcing rounds.

So the first response is at a maximum total context length of 4k, the second response (2nd index in the array) is a continuation of that last response up to a total of 8,192 tokens.

# Column Details

| Column | Description |
|--------|-------------|
| `question` | The question we want the model to answer |
| `answer` | The string answer |
| `task` | The name of the task the row belongs to |
| `prompt` | The prompt we will feed into the model to solve the question |
| `model_responses` | An array of strings that the model generated to answer the prompt (usually size of 4 or 34 depending on the evaluation task) |
| `model_responses__eval_is_correct` | An array aligned with `model_responses` containing booleans: `True` when the response was correct, `False` when incorrect or no answer was found |
| `model_responses__eval_extracted_answers` | An array aligned with `model_responses` containing the extracted answer strings from each response (usually the last answer in `<answer>` tags) |
| `model_responses__internal_answers__eval_is_correct` | An array aligned with `model_responses` where each value is an array of booleans for the correctness of intermediate answers within a trace |
| `model_responses__internal_answers__eval_extracted_answers` | Similar to `model_responses__eval_extracted_answers` but for internal/intermediate answers |
| `all_other_columns` | A catch-all column for additional task-dependent information (e.g., for countdown: target number and arguments) |
| `metadata` | Metadata about the question (alternative location for task-specific data like countdown target/arguments) |
| `prompt__metadata` | Metadata for the vLLM network request including URL and generation parameters. We used a customized [Curator](https://github.com/bespokelabsai/curator) to send raw text to `/completion` instead of `/chat/completion` for warm-start prompts and budget forcing |
| `model_responses__metadata` | Metadata returned from the vLLM request |

**Additional task-specific columns:** `answer_index`, `answer_key`, `choices`, `id`, `difficulty`, `domain`, `evaluation_type`, `expected_answer_format`, `original_answer`, `source`, `task_type`, `variant`, `acronym`, `formed_acronym`, `word_count`, `words`, `length`, `letters`

许可证：MIT
任务类别：
- 文本生成
语言：
- 英语
标签：
- 评估
- 技能工厂（skill-factory）

本数据集与评估数据集完全一致，仅`model_responses`数组为预算强制（budget forcing）轮次。

首个模型响应的最大总上下文长度为4k（即4096）Token，数组中第二个响应（索引为2）则是前一段响应的延续，总长度可达8192个Token。

# 列详情

| 列名 | 描述 |
|--------|-------------|
| `question` | 待模型作答的问题 |
| `answer` | 标准答案字符串 |
| `task` | 当前数据行所属的任务名称 |
| `prompt` | 用于喂入模型以求解该问题的提示词 |
| `model_responses` | 模型为响应该提示词所生成的字符串数组（根据评估任务的不同，数组长度通常为4或34） |
| `model_responses__eval_is_correct` | 与`model_responses`对齐的布尔值数组：当模型响应正确时为`True`，响应错误或未找到有效答案时为`False` |
| `model_responses__eval_extracted_answers` | 与`model_responses`对齐的数组，存储从每个模型响应中提取的答案字符串（通常为`<answer>`标签内的最后一段答案） |
| `model_responses__internal_answers__eval_is_correct` | 与`model_responses`对齐的数组，其中每个元素为子布尔数组，用于标记推理轨迹内中间答案的正确性 |
| `model_responses__internal_answers__eval_extracted_answers` | 与`model_responses__eval_extracted_answers`类似，但针对内部/中间答案 |
| `all_other_columns` | 通用兜底列，用于存储与任务相关的额外信息（例如在倒计时任务中存储目标数字与参数） |
| `metadata` | 与问题相关的元数据（可替代存储任务专属数据，例如倒计时任务的目标数字与参数） |
| `prompt__metadata` | vLLM网络请求的元数据，包含请求地址与生成参数。我们使用定制化的[Curator](https://github.com/bespokelabsai/curator)工具，将原始文本发送至`/completion`接口而非`/chat/completion`，以实现预热提示与预算强制（budget forcing） |
| `model_responses__metadata` | vLLM请求返回的元数据 |

**附加任务专属列：** `answer_index`、`answer_key`、`choices`、`id`、`difficulty`、`domain`、`evaluation_type`、`expected_answer_format`、`original_answer`、`source`、`task_type`、`variant`、`acronym`、`formed_acronym`、`word_count`、`words`、`length`、`letters`