Gen-Verse/LiveBench-ReasonFlux

---
license: mit
---

We use Stdio input/output format here. For example, for the task to calculate the sum of a list, the input and output are in the following format:
```python
input = "5\n1 2 3 4 5\n"
output = "15"
```
CodeContests and CodeForces are using this format, however, MBPP and part of LiveCodeBench are using functional input/output format, such like
```python
assert sum_function([1, 2, 3, 4, 5]) == 15
```
In this project, we have converted the the functional format to the Stdio format to achieve consistency.

[Paper](https://arxiv.org/abs/2506.03136) | [Code](https://github.com/Gen-Verse/CURE)

# Citation

```
@article{wang2025cure,
title={Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{white2024livebench,
title={Livebench: A challenging, contamination-free llm benchmark},
author={White, Colin and Dooley, Samuel and Roberts, Manley and Pal, Arka and Feuer, Ben and Jain, Siddhartha and Shwartz-Ziv, Ravid and Jain, Neel and Saifullah, Khalid and Naidu, Siddartha and others},
journal={arXiv preprint arXiv:2406.19314},
year={2024}
}
```

---
许可证：MIT
---

本数据集采用标准输入输出（Standard Input/Output，Stdio）格式。以列表求和任务为例，其输入与输出格式如下：
python
input = "5
1 2 3 4 5
"
output = "15"

CodeContests与CodeForces均采用该格式；而MBPP与部分LiveCodeBench则采用函数式输入输出格式，示例如下：
python
assert sum_function([1, 2, 3, 4, 5]) == 15

本项目已将函数式输入输出格式转换为标准输入输出格式，以实现统一的交互标准。

「论文」（https://arxiv.org/abs/2506.03136） | 「代码仓库」（https://github.com/Gen-Verse/CURE）

# 引用

@article{wang2025cure,
title={基于强化学习实现大语言模型（Large Language Model，LLM）编程器与单元测试器的协同演化},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{white2024livebench,
title={LiveBench：一款高难度、无数据污染的大语言模型基准测试集},
author={White, Colin and Dooley, Samuel and Roberts, Manley and Pal, Arka and Feuer, Ben and Jain, Siddhartha and Shwartz-Ziv, Ravid and Jain, Neel and Saifullah, Khalid and Naidu, Siddartha and others},
journal={arXiv preprint arXiv:2406.19314},
year={2024}
}