copenlu/cmt-benchmark-counterfact

CounterFact数据集是cmt-benchmark项目的一部分，基于Meng等人于2022年提出的流行CounterFact数据集。该数据集从Pythia 6.9B的参数记忆中采样了899个CounterFact样本，并包含与GPT-2 XL参数记忆匹配的546个样本。数据集包含两种版本：gpt2-xl和pythia-6.9b，每个版本都有对应的验证集和测试集。数据集的样本基于（主题、关系、对象）事实三元组，要求模型预测相应的对象。数据集包括多个列，其中一些列在不同版本之间是相同的，例如样本ID、关系ID、主题、上下文类型、模板等，而其他列则依赖于数据集的版本，例如模型预测和概率。

CounterFact dataset is a part of the cmt-benchmark project, based on the popular CounterFact dataset proposed by Meng et al. in 2022. The dataset samples 899 CounterFact instances from the parametric memory of Pythia 6.9B and includes 546 samples that match the parametric memory of GPT-2 XL. The dataset comes in two versions: gpt2-xl and pythia-6.9b, each with corresponding validation and test sets. The samples in the dataset are based on (subject, relation, object) fact triplets and require the model to predict the corresponding object. The dataset includes multiple columns, some of which are identical across different versions, such as sample id, relation id, subject, context type, template, etc., while others depend on the version of the dataset, such as model predictions and probabilities.