shivaniachary123/pii-masking-200k

Ai4Privacy PII200k数据集是一个用于训练模型从文本中移除个人可识别信息（PII）的大规模开源数据集，特别针对AI助手和大型语言模型（LLMs）的隐私保护场景。数据集包含约209,000个示例，总计13.6百万文本标记和649,000个PII标记，覆盖英语、法语、德语和意大利语四种语言。数据通过专有算法合成生成，确保无隐私侵犯，并经过人工验证以保证高质量。数据集涵盖54个PII类别（如姓名、地址、电话号码等），针对229个讨论主题/用例，分布在商业、教育、心理学和法律领域，并包含5种交互风格（如日常对话、正式文档、电子邮件等）。每个示例以JSON对象形式提供，包括原始文本（含PII）、脱敏后文本（PII被替换为类别标签）、隐私掩码映射（标注PII的位置和类别）、跨度标签、MBERT BIO标签和BERT标记化文本等字段。数据集主要用于令牌分类和文本生成任务，支持多种预训练模型（如BERT、RoBERTa、T5等），适用于聊天机器人、客户支持系统、数据匿名化等多种隐私保护应用。

The Ai4Privacy PII200k Dataset is a large-scale open dataset designed to train models for removing personally identifiable information (PII) from text, particularly in the context of AI assistants and large language models (LLMs). It contains approximately 209,000 examples with 13.6 million text tokens and 649,000 PII tokens, covering four languages: English, French, German, and Italian. The data is synthetically generated using proprietary algorithms to avoid privacy violations and has been human-in-the-loop validated for high quality. The dataset includes 54 PII classes (e.g., names, addresses, phone numbers) targeting 229 discussion subjects/use cases across business, education, psychology, and legal fields, with 5 interaction styles (e.g., casual conversation, formal documents, emails). Each example is provided as a JSON object with fields such as source_text (unmasked text with PII), target_text (PII-free masked text), privacy_mask (mapping of PII instances to labels), span_labels, mbert_bio_labels, and tokenized text. It is compatible with machine learning tasks like token classification and text generation, supporting various pre-trained models (e.g., BERT, RoBERTa, T5), and is applicable to use cases such as chatbots, customer support systems, and data anonymization for privacy protection.