LiveCodeBench-ReasonFlux

We use Stdio input/output format here. For example, for the task to calculate the sum of a list, the input and output are in the following format:
```python
input = "5\n1 2 3 4 5\n"
output = "15"
```
CodeContests and CodeForces are using this format, however, MBPP and part of LiveCodeBench are using functional input/output format, such like
```python
assert sum_function([1, 2, 3, 4, 5]) == 15
```
In this project, we have converted the the functional format to the Stdio format to achieve consistency.

[Paper](https://arxiv.org/abs/2506.03136) | [Code](https://github.com/Gen-Verse/CURE)

# Citation

```
@article{wang2025cure,
title={Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{jain2024livecodebench,
title={Livecodebench: Holistic and contamination free evaluation of large language models for code},
author={Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion},
journal={arXiv preprint arXiv:2403.07974},
year={2024}
}
```

本数据集采用标准输入输出（Stdio）格式。以计算列表求和任务为例，其输入与输出格式如下：
python
input = "5
1 2 3 4 5
"
output = "15"

CodeContests与CodeForces平台均采用此类格式，而MBPP与部分LiveCodeBench则采用函数式输入输出格式，例如：
python
assert sum_function([1, 2, 3, 4, 5]) == 15

本项目已将函数式输入输出格式转换为标准输入输出格式，以实现全数据集格式统一。

[论文](https://arxiv.org/abs/2506.03136) | [代码](https://github.com/Gen-Verse/CURE)

# 引用文献

@article{wang2025cure,
title={基于强化学习的大语言模型（Large Language Model）程序员与单元测试器协同进化},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{jain2024livecodebench,
title={LiveCodeBench：面向代码任务的全面无数据污染大语言模型评估},
author={Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion},
journal={arXiv preprint arXiv:2403.07974},
year={2024}
}

搜集汇总

背景与挑战

背景概述

LiveCodeBench-ReasonFlux是一个用于代码评估的数据集，它通过将功能性输入/输出格式转换为Stdio格式来确保数据一致性。该数据集支持相关研究，并引用了关于大型语言模型代码评估和强化学习方法的论文。