Qwen3-235B-A22B-nonreasoning

# qwen-235b-22a-nonreasoning Evaluation Results

## Summary

| Benchmark | Score | Metric | Samples | Overlong rate |
|-----------|-------|--------|---------|---------------|
| aime24 | 0.341 | math_pass@1:64_samples | 64 | 0.0% |
| aime25 | 0.251 | math_pass@1:64_samples | 64 | 0.0% |
| arenahard | 0.917 | eval/overall_winrate | 500 | 0.0% |
| bbh_generative | 0.860 | extractive_match | 1 | 0.0% |
| creative-writing-v3 | 0.741 | creative_writing_score | 96 | 0.0% |
| drop_generative_nous | 0.794 | drop_acc | 1 | 0.0% |
| eqbench3 | 0.811 | eqbench_score | 135 | 0.0% |
| gpqa_diamond | 0.577 | gpqa_pass@1:8_samples | 8 | 0.0% |
| ifeval | 0.912 | inst_level_loose_acc | 1 | 0.0% |
| lcb-v6-aug2024+ | 0.346 | eval/pass_1 | 1 | 0.0% |
| math_500 | 0.903 | math_pass@1:4_samples | 4 | 0.0% |
| mmlu_generative | 0.865 | extractive_match | 1 | 0.0% |
| mmlu_pro | 0.755 | pass@1:1_samples | 1 | 0.0% |
| musr_generative | 0.648 | extractive_match | 1 | 0.0% |
| obqa_generative | 0.964 | extractive_match | 1 | 0.0% |
| rewardbench | 0.692 | eval/percent_correct | 1 | 0.0% |
| simpleqa_nous | 0.078 | fuzzy_match | 1 | 0.0% |

Overlong rate: 0 / 64,523 samples (0.0%) missing closing `</think>` tag

## Detailed Results

### aime24

| Metric | Score | Std Error |
|--------|-------|----------|
| math_pass@1:1_samples | 0.367 | 0.089 |
| math_pass@1:4_samples | 0.383 | 0.082 |
| math_pass@1:8_samples | 0.388 | 0.077 |
| math_pass@1:16_samples | 0.354 | 0.074 |
| math_pass@1:32_samples | 0.343 | 0.073 |
| math_pass@1:64_samples | 0.341 | 0.071 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:35:15
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 1920)

### aime25

| Metric | Score | Std Error |
|--------|-------|----------|
| math_pass@1:1_samples | 0.167 | 0.069 |
| math_pass@1:4_samples | 0.242 | 0.070 |
| math_pass@1:8_samples | 0.250 | 0.063 |
| math_pass@1:16_samples | 0.263 | 0.065 |
| math_pass@1:32_samples | 0.259 | 0.066 |
| math_pass@1:64_samples | 0.251 | 0.065 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:28:42
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 1920)

### arenahard

| Metric | Score | Std Error |
|--------|-------|----------|
| eval/overall_winrate | 0.917 | 0.000 |
| eval/total_samples | 500.000 | 0.000 |
| eval/win_count | 443.000 | 0.000 |
| eval/tie_count | 31.000 | 0.000 |
| eval/loss_count | 26.000 | 0.000 |
| eval/win_rate | 0.886 | 0.000 |
| eval/tie_rate | 0.062 | 0.000 |
| eval/loss_rate | 0.052 | 0.000 |
| eval/winrate_arena-hard-v0.1 | 0.917 | 0.000 |

**Model:** qwen3-235b-nothink-arena
**Evaluation Time (hh:mm:ss):** 00:12:21
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 500)

### bbh_generative

| Metric | Score | Std Error |
|--------|-------|----------|
| extractive_match | 0.860 | 0.018 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:39:08
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 5511)

### creative-writing-v3

| Metric | Score | Std Error |
|--------|-------|----------|
| creative_writing_score | 0.741 | 0.000 |
| num_samples | 96.000 | 0.000 |

**Model:** qwen3-235b-a22b-nonthinking-eq
**Evaluation Time (hh:mm:ss):** N/A
**Temperature:** N/A
**Overlong samples:** 0.0% (0 / 96)

### drop_generative_nous

| Metric | Score | Std Error |
|--------|-------|----------|
| drop_acc | 0.794 | 0.004 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:37:28
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 9536)

### eqbench3

| Metric | Score | Std Error |
|--------|-------|----------|
| eqbench_score | 0.811 | 0.000 |
| num_samples | 135.000 | 0.000 |

**Model:** qwen3-235b-nothink-arena
**Evaluation Time (hh:mm:ss):** N/A
**Temperature:** N/A
**Overlong samples:** 0.0% (0 / 135)

### gpqa_diamond

| Metric | Score | Std Error |
|--------|-------|----------|
| gpqa_pass@1:1_samples | 0.566 | 0.035 |
| gpqa_pass@1:4_samples | 0.576 | 0.027 |
| gpqa_pass@1:8_samples | 0.577 | 0.026 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:29:13
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 1584)

### ifeval

| Metric | Score | Std Error |
|--------|-------|----------|
| prompt_level_strict_acc | 0.826 | 0.016 |
| inst_level_strict_acc | 0.882 | 0.000 |
| prompt_level_loose_acc | 0.867 | 0.015 |
| inst_level_loose_acc | 0.912 | 0.000 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:21:01
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 541)

### lcb-v6-aug2024+

| Metric | Score | Std Error |
|--------|-------|----------|
| eval/pass_1 | 0.346 | 0.000 |
| eval/easy_pass_1 | 0.881 | 0.000 |
| eval/medium_pass_1 | 0.327 | 0.000 |
| eval/hard_pass_1 | 0.071 | 0.000 |
| eval/completion_length | 2029.859 | 0.000 |

**Model:** qwen-235ba22-reasoning
**Evaluation Time (hh:mm:ss):** 00:43:49
**Temperature:** N/A
**Overlong samples:** 0.0% (0 / 7264)

### math_500

| Metric | Score | Std Error |
|--------|-------|----------|
| math_pass@1:1_samples | 0.908 | 0.013 |
| math_pass@1:4_samples | 0.903 | 0.011 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:17:23
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 2000)

### mmlu_generative

| Metric | Score | Std Error |
|--------|-------|----------|
| extractive_match | 0.865 | 0.003 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:55:28
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 14042)

### mmlu_pro

| Metric | Score | Std Error |
|--------|-------|----------|
| pass@1:1_samples | 0.755 | 0.004 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 01:10:21
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 12032)

### musr_generative

| Metric | Score | Std Error |
|--------|-------|----------|
| extractive_match | 0.648 | 0.029 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:03:10
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 756)

### obqa_generative

| Metric | Score | Std Error |
|--------|-------|----------|
| extractive_match | 0.964 | 0.008 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:02:01
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 500)

### rewardbench

| Metric | Score | Std Error |
|--------|-------|----------|
| eval/percent_correct | 0.692 | 0.000 |
| eval/total_samples | 1865.000 | 0.000 |
| eval/correct_samples | 1290.000 | 0.000 |
| eval/format_compliance_rate | 0.961 | 0.000 |
| eval/avg_response_length | 2379.403 | 0.000 |
| eval/response_length_std | 740.237 | 0.000 |
| eval/judgment_entropy | 1.496 | 0.000 |
| eval/most_common_judgment_freq | 0.283 | 0.000 |
| eval/format_error_rate | 0.041 | 0.000 |
| eval/avg_ties_rating | 4.042 | 0.000 |
| eval/ties_error_rate | 0.003 | 0.000 |
| eval/percent_correct_Factuality | 0.577 | 0.000 |
| eval/percent_correct_Precise IF | 0.356 | 0.000 |
| eval/percent_correct_Math | 0.661 | 0.000 |
| eval/percent_correct_Safety | 0.733 | 0.000 |
| eval/percent_correct_Focus | 0.826 | 0.000 |
| eval/percent_correct_Ties | 0.971 | 0.000 |
| eval/choice_samples | 1763.000 | 0.000 |
| eval/ties_samples | 102.000 | 0.000 |
| eval/choice_format_compliance_rate | 0.959 | 0.000 |
| eval/ties_format_compliance_rate | 1.000 | 0.000 |
| eval/wrong_answer_a_bias_rate | 0.311 | 0.000 |
| eval/wrong_answer_total_count | 572.000 | 0.000 |
| eval/wrong_answer_a_count | 178.000 | 0.000 |

**Model:** qwen3-235b-reward-redo-nothink
**Evaluation Time (hh:mm:ss):** 00:30:06
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 1865)

### simpleqa_nous

| Metric | Score | Std Error |
|--------|-------|----------|
| exact_match | 0.055 | 0.003 |
| fuzzy_match | 0.078 | 0.004 |

**Model:** qwen-235b-22a-nonreasoning
**Evaluation Time (hh:mm:ss):** 00:16:58
**Temperature:** 0.6
**Overlong samples:** 0.0% (0 / 4321)