Minicode

该数据集名为Minicode，它是一个评估代码代理能力的基准，要求这些代理将多个独立的解决方案最小化和重构为一个联合库，以此来测试代码代理在重构代码以支持增长和可重用性方面的能力。该数据集通过聚类方法被划分为不重叠的集合，为评估代码代理在压缩和正确性方面的性能提供了基准。该数据集的规模为从大约9千个问题中筛选出的4596个Python解决方案，其任务重点是代码重构和库生成。This dataset is named Minicode, which is a benchmark for evaluating code agent capabilities. It requires agents to minimize and refactor multiple independent solutions into a unified library, thereby testing the code agents' ability to refactor code to support scalability and reusability. The dataset is divided into non-overlapping sets via clustering methods, providing a benchmark for assessing code agents' performance in terms of code compression and correctness. Comprising 4,596 Python solutions screened from approximately 9,000 problems, its task focuses on code refactoring and library generation.

搜集汇总

背景与挑战

背景概述

Minicode是一个评估代码代理能力的基准数据集，专注于代码重构和库生成任务。它通过聚类方法将4596个Python解决方案划分为不重叠的集合，要求代理将多个独立方案最小化和重构为联合库，以测试其在支持增长和可重用性方面的性能。该数据集为评估代码代理在压缩和正确性方面的能力提供了标准化基准。