FVEval

该数据集名为FVEval，是首个全面评估框架，旨在表征大型语言模型（LLM）在数字硬件设计中的形式验证（FV）相关任务上的性能。该基准测试包含了生成SystemVerilog断言和对设计寄存器传输级（RTL）进行推理的任务，并提供全面和部分功能等价的度量标准。该数据集涵盖了广泛现有的LLM，包括专有和开源模型。其任务之一是评估语言模型根据自然语言描述生成SystemVerilog断言的能力。

This dataset, named FVEval, is the first comprehensive evaluation framework aimed at characterizing the performance of Large Language Models (LLMs) on tasks related to formal verification (FV) in digital hardware design. This benchmark encompasses tasks of generating SystemVerilog assertions and conducting reasoning on design Register-Transfer Level (RTL) circuits, and provides both comprehensive and partial functional equivalence metrics. This dataset covers a wide range of existing LLMs, including both proprietary and open-source models. One of its tasks is to evaluate the capability of language models to generate SystemVerilog assertions based on natural language descriptions.