nhagar/CC-MAIN-2020-16_urls

这是一个包含网页抓取信息的数据集，具体特征包括网页的抓取内容（crawl）、网址的主机名（url_host_name）以及每个主机名的网址数量（url_count）。数据集提供了训练集分割，共有约6000万个样本，数据集总大小约为2.9GB。This is a dataset containing web crawling information, with features including the content of the web page (crawl), the host name of the URL (url_host_name), and the number of URLs for each host name (url_count). The dataset provides a training set split, with a total of approximately 60 million samples, and the total size of the dataset is about 2.9GB.