SPoC (Pseudocode-to-Code)

我们考虑将伪代码映射到功能正确的长程序的任务。给定测试用例作为验证程序的机制，我们搜索伪代码的可能翻译空间以找到通过验证的程序。然而，如果没有适当的信用分配来定位程序失败的根源，就很难将搜索引导到更有希望的程序上。我们建议根据编译错误的信号执行信用分配，编译错误占程序失败的 88.7%。具体来说，我们将每个伪代码行的翻译视为程序的一个离散部分，并且每当合成程序编译失败时，错误定位方法会尝试识别导致失败的程序部分。然后，我们将搜索重点放在这些部分的伪代码的替代翻译上。为了评估，我们收集了 SPoC 数据集（基于搜索的伪代码到代码），其中包含 18,356 个程序，其中包含人工编写的伪代码和测试用例。在 100 个程序编译的预算下，与使用伪代码的 top-one 翻译相比，执行搜索将合成成功率从 25.6% 提高到 44.7%。We consider the task of mapping pseudocode to functionally correct long programs. Given test cases as the mechanism to validate programs, we search the space of possible translations of the pseudocode to find validated programs. However, without proper credit assignment to locate the root cause of program failures, it is difficult to guide the search toward more promising programs. We propose to perform credit allocation based on signals from compilation errors, which account for 88.7% of program failures. Specifically, we treat the translation of each pseudocode line as a discrete component of the program. Whenever a synthesized program fails compilation, the error localization method attempts to identify the program components that caused the failure, and then we focus the search on alternative translations of the pseudocode corresponding to these components. For evaluation, we collected the SPoC dataset (Search-based Pseudocode to Code), which contains 18,356 programs with human-written pseudocode and test cases. Under a budget of 100 program compilations, the proposed search improves the synthesis success rate from 25.6% to 44.7% compared to using the top-one translation of the pseudocode.

搜集汇总

背景与挑战

背景概述

SPoC是一个用于伪代码到代码转换的数据集，包含18,356个带有伪代码和测试用例的程序。该数据集通过基于编译错误的搜索方法，显著提升了程序合成的成功率。