raca-workspace-v1/algo-sft-eval-traces-conlang-morphology-ordered-rules-d5d7-v4

---
license: mit
tags:
- algo-sft-eval-redo
- conlang_morphology
- algo
---

# algo-sft-eval-traces-conlang-morphology-ordered-rules-d5d7-v4

Full eval traces for algo-sft-conlang-morphology-ordered-rules-d5d7 across test/harder/ood splits

## Dataset Info

- **Rows**: 2000
- **Columns**: 11

## Columns

| Column | Type | Description |
|--------|------|-------------|
| question_id | Value('string') | Unique question identifier from eval set |
| split | Value('string') | Evaluation split: test (in-distribution), harder (scaled up), ood (structural out-of-distribution) |
| domain | Value('string') | Task domain: formal_logic, conlang_morphology, cellular_automata, long_arithmetic |
| task | Value('string') | Specific task variant (e.g., formal_logic_bottom_up) |
| prompt | Value('string') | Full prompt sent to the model |
| model_response | Value('string') | Complete untruncated model output |
| extracted_answer | Value('string') | Answer extracted by domain-specific parser |
| ground_truth | Value('string') | Expected correct answer |
| correct | Value('bool') | Whether extracted_answer matched ground_truth |
| finish_reason | Value('string') | vLLM finish reason: stop (natural end) or length (hit max_tokens) |
| token_count | Value('int64') | Number of tokens in model_response |

## Generation Parameters

```json
{
"script_name": "eval_model.py",
"model": "reasoning-degeneration-dev/algo-sft-conlang-morphology-ordered-rules-d5d7",
"description": "Full eval traces for algo-sft-conlang-morphology-ordered-rules-d5d7 across test/harder/ood splits",
"hyperparameters": {
"max_tokens": 32768,
"max_model_len": 32768,
"temperature": 0.0,
"base_model": "Qwen/Qwen2.5-1.5B-Instruct"
},
"input_datasets": []
}
```

## Usage

```python
from datasets import load_dataset

dataset = load_dataset("raca-workspace-v1/algo-sft-eval-traces-conlang-morphology-ordered-rules-d5d7-v4", split="train")
print(f"Loaded {len(dataset)} rows")
```

---

*Uploaded via [RACA](https://github.com/Zayne-sprague/Dr-Claude-Code) hf_utility.*