HPAI-BSC/NotSoTiny-25-12

NotSoTiny是一个大型、结构丰富且活的基准测试集，旨在评估大型语言模型(LLM)在生成上下文感知的RTL(寄存器传输级)代码方面的能力。该数据集基于Tiny Tapeout社区生产的数百个真实硬件设计构建，通过定期纳入新设计来克服先前静态数据集的局限性，使其对数据污染具有弹性。与之前依赖独立模块或明确规范的基准不同，NotSoTiny专注于上下文模块完成，其中模型被呈现一个完整的设计上下文，其中一个模块被屏蔽。LLM必须仅从周围的实现中推断缺失模块的功能和接口，这反映了现实世界开发场景，即新组件必须集成到现有系统中。该数据集包括25-12版本，包含1,114个经过重复数据删除和整理的任务，这些任务源自真实的、已流片的硬件设计，使其比现有的RTL基准测试集更大、更复杂。

NotSoTiny is a large, structurally rich, and living benchmark designed to assess Large Language Models (LLMs) on the generation of context-aware RTL (Register-Transfer Level) code. Built from hundreds of real hardware designs produced by the Tiny Tapeout community, this benchmark overcomes the limitations of prior static datasets by periodically incorporating new designs, making it resilient to data contamination. Unlike previous benchmarks which rely on standalone modules or explicit specifications, NotSoTiny focuses on contextual module completion. In this setup, models are presented with a full design context, where one module is masked. The LLM must infer the missing modules functionality and interface solely from the surrounding implementation, mirroring real-world development scenarios where new components must integrate into existing systems. This dataset includes the 25-12 release, with 1,114 deduplicated and curated tasks derived from real, taped-out hardware designs, making it significantly larger and more complex than existing RTL benchmarks.