KLoB

KLoB是一个用于评估语言模型中知识定位方法可靠性的基准，它检查了知识定位方法应满足的三个基本属性，并提供了一种重新评估事实知识局部性假设有效性的方法。

KLoB is a benchmark for evaluating the reliability of knowledge localization methods in language models. It examines three fundamental properties that knowledge localization methods should satisfy, and provides a method to re-evaluate the validity of the factual knowledge locality hypothesis.

原始信息汇总

KLoB

数据集名称：KLoB 数据集描述：用于评估语言模型中知识定位方法的基准数据集。

数据集示例

KLoB-c

示例内容： json { "prompts": ["The native language of Norman Rockwell is _", "The mother tongue of Norman Rockwell is _", "The language Norman Rockwell speaks at hometown is "], "knowledge": ["Norman Rockwell", "native(P103)", "English"] }

KLoB-r

示例内容： json { "single_knowledge": ["Ann Druyan", "spouse_(P26)", "Carl Sagan"], "knowledge_chain": ["Ann Druyan", "spouse_(P26)", "Carl Sagan", "employer_(P108)", "Cornell University"], "single_knowlege_prompts": ["Ann Druyan is married to _", "The spouse of Ann Druyan is none other than _", "In matrimony, Ann Druyan is bound to _"], "knowlege_chain_prompt": "the employer of the spouse of Ann Druyan is _" }

KLoB-u

示例内容：

"extirpation sensa subiodide conquedle spikebill drabbish unvalidating becuiba _"