LiveCodeBench

We use Stdio input/output format here. For example, for the task to calculate the sum of a list, the input and output are in the following format:
```python
input = "5\n1 2 3 4 5\n"
output = "15"
```
CodeContests and CodeForces are using this format, however, MBPP and part of LiveCodeBench are using functional input/output format, such like
```python
assert sum_function([1, 2, 3, 4, 5]) == 15
```
In this project, we have converted the the functional format to the Stdio format to achieve consistency.

[Paper](https://arxiv.org/abs/2506.03136) | [Code](https://github.com/Gen-Verse/CURE)

# Citation

```
@article{wang2025cure,
title={Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{jain2024livecodebench,
title={Livecodebench: Holistic and contamination free evaluation of large language models for code},
author={Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion},
journal={arXiv preprint arXiv:2403.07974},
year={2024}
}
```

本研究采用标准输入输出（Stdio）格式。以计算列表求和任务为例，其输入与输出格式如下：
python
input = "5
1 2 3 4 5
"
output = "15"

CodeContests与CodeForces均采用该格式；而MBPP与部分LiveCodeBench则使用函数式输入输出格式，示例如下：
python
assert sum_function([1, 2, 3, 4, 5]) == 15

本项目已将函数式输入输出格式转换为标准输入输出格式，以保证格式统一。
[论文](https://arxiv.org/abs/2506.03136) | [代码](https://github.com/Gen-Verse/CURE)

# 引用

@article{wang2025cure,
title={Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning},
author={Wang, Yinjie and Yang, Ling and Tian, Ye and Shen, Ke and Wang, Mengdi},
journal={arXiv preprint arXiv:2506.03136},
year={2025}
}

@article{jain2024livecodebench,
title={Livecodebench: Holistic and contamination free evaluation of large language models for code},
author={Jain, Naman and Han, King and Gu, Alex and Li, Wen-Ding and Yan, Fanjia and Zhang, Tianjun and Wang, Sida and Solar-Lezama, Armando and Sen, Koushik and Stoica, Ion},
journal={arXiv preprint arXiv:2403.07974},
year={2024}
}

搜集汇总

背景与挑战

背景概述

LiveCodeBench是一个尚未更新内容的数据集，采用MIT许可证，当前大小为4.13GB，下载次数为607次。