NicheDataSolutions/multilingual-pii-free-dataset

NicheDataSolutions多语言PII-Free数据集v2包含484个高质量文档(约1800万token)，涵盖5个专业领域和12种语言。数据集经过彻底的个人身份信息(PII)清除处理，可用于商业AI训练。主要特点包括：100%绿色许可证(CC0/美国公共领域/政府公共领域)；采用智能PII移除技术(正则表达式+多语言SpaCy NER)；包含超稀有语言数据(白俄罗斯语97篇、乌尔都语33篇、挪威语38篇、丹麦语20篇)；领域多样化(科学187篇、法律162篇、金融70篇、医学45篇、IT遗留系统5篇)。数据格式为JSONL，每行包含metadata(领域、语言、质量评分等元数据)和content(清洗后的训练文本)字段。适用于多语言基础模型预训练、领域特定AI(法律AI、医疗AI、金融AI)微调以及低资源语言评估基准。

NicheDataSolutions — Multilingual PII-Free Dataset v2 contains 484 high-quality documents (~18M tokens) across 5 niches and 12 languages, fully PII-scrubbed and cleared for commercial AI training. Key features include: 100% GREEN License (CC0 / US Public Domain / Government PD); Smart PII Removal (Regex + Multilingual SpaCy NER); Ultra-Rare Languages (Belarusian - 97 docs, Urdu - 33, Norwegian - 38, Danish - 20); Niche Diverse (Science - 187, Legal - 162, Finance - 70, Medical - 45, IT Legacy - 5). Format is JSONL with each line containing metadata (niche, language, quality_score etc.) and content (cleaned training text). Ideal for multilingual base model pre-training, domain-specific fine-tuning (Legal AI, Medical AI, Finance AI), and low-resource language evaluation benchmarks.