maisonOP/spia

SPIA是第一个用于文本匿名化中主题级隐私评估的基准。与现有的专注于单目标或基于跨度的评估方法不同，SPIA能够捕捉文档中所有数据主体的基于推理的隐私风险。数据集包含来自欧洲人权法院（ECHR）的法律文件和合成的在线内容，专家注释者识别了每个文档中的所有数据主体，并注释了个人可识别信息（PII）、推理的确定性（1-5级）和推理的难度（1-5级）。数据集包含15种PII类别，分为CODE（5种）和NON-CODE（10种）两大类。数据集结构以JSON格式呈现，包含元数据、文本和主体信息。

SPIA is the first benchmark for subject-level privacy assessment in text anonymization. Unlike existing methods that focus on single-target or span-based evaluation, SPIA captures inference-based privacy risks across all data subjects in a document. The dataset includes legal documents from the European Court of Human Rights (ECHR) and synthetic online content. Expert annotators identified all data subjects in each document and annotated personal identifiable information (PIIs), certainty of inference (1-5 scale), and hardness of inference (1-5 scale). The dataset covers 15 PII categories, divided into CODE (5 types) and NON-CODE (10 types). The dataset structure is presented in JSON format, including metadata, text, and subject information.