alzoubi36/privaseer

Privaseer数据集是一个包含大量从网络上收集的隐私政策文本的语料库，旨在促进大规模半监督和无监督模型的创建，以解释和简化隐私政策。该数据集包含1,005,380个英文网站隐私政策，涵盖了比现有公开隐私政策语料库多十倍的独特网站。数据集创建过程包括网络爬虫、语言检测、文档分类、重复和近重复内容去除以及内容提取等步骤。此外，该数据集还用于预训练PrivBERT，一个基于Transformer的隐私政策语言模型，并在数据实践分类和问答任务中取得了最先进的结果。

The Privaseer Dataset is a corpus of privacy policy texts collected from the web, designed to facilitate the development of large-scale semi-supervised and unsupervised models for interpreting and simplifying privacy policies. This dataset includes 1,005,380 privacy policies from English-language websites, covering ten times as many unique websites as existing public privacy policy corpora. The construction process of the dataset involves steps such as web crawling, language detection, document classification, removal of duplicate and near-duplicate content, and content extraction. Furthermore, this dataset has been utilized to pre-train PrivBERT, a Transformer-based language model tailored for privacy policies, which has achieved state-of-the-art performance on data practice classification and question answering tasks.

原始信息汇总

数据集概述

数据集信息

• 名称: Privaseer Dataset
• 版本: Huggingface版本
• 许可证: GPL-3.0

数据集特征

• hash: 字符串类型
• url: 字符串类型
• text: 字符串类型
• title: 字符串类型
• id: 整数类型（int64）

数据集拆分

• 训练集:
  • 大小: 17080868768字节
  • 示例数量: 2180300

数据集大小

• 下载大小: 8133175578字节
• 数据集总大小: 17080868768字节