SAFIM

该数据集名为SAFIM，它是一个评估大型语言模型在代码填空任务上的基准，专注于对程序结构如代码块和条件表达式的语法感知完成。该数据集包含了来自多种编程语言的17,720个示例，这些示例均源自2022年4月之后的最新代码提交。SAFIM还提供了一个健壮的框架，其中包括多种提示设计和新颖的语法感知后处理技术，以促进LLM之间的准确性和公平比较。规模上，该数据集包含了17,720个示例，任务类型为代码填空（Fim）。

This dataset, named SAFIM, is a benchmark for evaluating large language models (LLMs) on code fill-in-the-middle (FIM) tasks, focusing on syntax-aware completion of program structures such as code blocks and conditional expressions. It consists of 17,720 examples sourced from recent code commits made after April 2022 across multiple programming languages. SAFIM also provides a robust framework that includes diverse prompt design strategies and novel syntax-aware post-processing techniques, enabling accurate and fair comparative evaluations across different LLMs. In terms of scale, the dataset comprises 17,720 examples, with the task type being code fill-in-the-middle (FIM).