INJ Benchmark, DEL Benchmark

本文提出了两个互补的基准测试：INJ Benchmark和DEL Benchmark，用于评估大型语言模型（LLMs）中定位记忆数据的能力。INJ Benchmark通过主动向LLM权重的小子集中注入新信息，直接评估定位方法是否能识别这些“真实”权重。DEL Benchmark则通过测量已识别神经元的丢失对预训练序列记忆的影响来评估定位。这两个基准测试从不同角度出发，但都得出了对五种定位方法的一致排名。

This paper proposes two complementary benchmarks, namely the INJ Benchmark and the DEL Benchmark, for evaluating the ability of large language models (LLMs) to localize memorized data. The INJ Benchmark directly assesses whether localization methods can identify these 'ground-truth' weights by actively injecting novel information into a small subset of LLM weights. The DEL Benchmark, by contrast, evaluates localization by measuring the impact of ablating the identified neurons on the memorization of pre-trained sequences. Although the two benchmarks approach the task from distinct perspectives, they both yield a consistent ranking of the five localization methods.