kth8/gpt-oss-20b-supergpqa-benchmark

---
license: apache-2.0
language:
- en
base_model: openai/gpt-oss-20b
datasets:
- m-a-p/SuperGPQA
---
Benchmark of [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b) against [m-a-p/SuperGPQA](https://huggingface.co/datasets/m-a-p/SuperGPQA) dataset.

Accuracy: 48.4% with Python tool.
| Metric | Value |
|----------------------|---------------|
| **Correct** | 484 |
| **Incorrect** | 516 |
| **Errors** | 0 |
| **Total samples** | 1000 |
| **Python tool calls**| 718 |
| **Total completion tokens** | 2,182,037 |

Raw stats:
```json
{
"accuracy": 0.484,
"correct": 484,
"incorrect": 516,
"error": 0,
"total": 1000,
"python_tool_calls": 718,
"completion_tokens": 2182037
}
```

许可证：Apache-2.0
语言：
- 英语
基础模型：openai/gpt-oss-20b
所用数据集：
- m-a-p/SuperGPQA

本基准测试针对[openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)模型与[m-a-p/SuperGPQA](https://huggingface.co/datasets/m-a-p/SuperGPQA)数据集开展性能对比评测。采用Python工具进行推理时，该模型准确率达48.4%。

| 评测指标 | 数值 |
|----------------------|---------------|
| **正确样本数** | 484 |
| **错误样本数** | 516 |
| **异常报错数** | 0 |
| **总样本量** | 1000 |
| **Python工具调用次数**| 718 |
| **总补全Token(Token)数** | 2,182,037 |

原始统计数据如下：
json
{
"accuracy": 0.484,
"correct": 484,
"incorrect": 516,
"error": 0,
"total": 1000,
"python_tool_calls": 718,
"completion_tokens": 2182037
}