newsgroup
收藏魔搭社区2025-11-27 更新2025-07-12 收录
下载链接：
https://modelscope.cn/datasets/google-research-datasets/newsgroup
下载链接
链接失效反馈官方服务：
资源简介：
# Dataset Card for "newsgroup"
## Table of Contents
- [Dataset Description](#dataset-description)
- [Dataset Summary](#dataset-summary)
- [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
- [Languages](#languages)
- [Dataset Structure](#dataset-structure)
- [Data Instances](#data-instances)
- [Data Fields](#data-fields)
- [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
- [Curation Rationale](#curation-rationale)
- [Source Data](#source-data)
- [Annotations](#annotations)
- [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Social Impact of Dataset](#social-impact-of-dataset)
- [Discussion of Biases](#discussion-of-biases)
- [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
- [Dataset Curators](#dataset-curators)
- [Licensing Information](#licensing-information)
- [Citation Information](#citation-information)
- [Contributions](#contributions)
## Dataset Description
- **Homepage:** [http://qwone.com/~jason/20Newsgroups/](http://qwone.com/~jason/20Newsgroups/)
- **Repository:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Paper:** [NewsWeeder: Learning to Filter Netnews](https://doi.org/10.1016/B978-1-55860-377-6.50048-7)
- **Point of Contact:** [More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **Size of downloaded dataset files:** 929.27 MB
- **Size of the generated dataset:** 124.41 MB
- **Total amount of disk used:** 1.05 GB
### Dataset Summary
The 20 Newsgroups data set is a collection of approximately 20,000 newsgroup documents, partitioned (nearly) evenly across
20 different newsgroups. To the best of my knowledge, it was originally collected by Ken Lang, probably for his Newsweeder:
Learning to filter netnews paper, though he does not explicitly mention this collection. The 20 newsgroups collection has become
a popular data set for experiments in text applications of machine learning techniques, such as text classification and text clustering.
does not include cross-posts and includes only the "From" and "Subject" headers.
### Supported Tasks and Leaderboards
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Languages
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## Dataset Structure
### Data Instances
#### 18828_alt.atheism
- **Size of downloaded dataset files:** 14.67 MB
- **Size of the generated dataset:** 1.67 MB
- **Total amount of disk used:** 16.34 MB
An example of 'train' looks as follows.
```
```
#### 18828_comp.graphics
- **Size of downloaded dataset files:** 14.67 MB
- **Size of the generated dataset:** 1.66 MB
- **Total amount of disk used:** 16.33 MB
An example of 'train' looks as follows.
```
```
#### 18828_comp.os.ms-windows.misc
- **Size of downloaded dataset files:** 14.67 MB
- **Size of the generated dataset:** 2.38 MB
- **Total amount of disk used:** 17.05 MB
An example of 'train' looks as follows.
```
```
#### 18828_comp.sys.ibm.pc.hardware
- **Size of downloaded dataset files:** 14.67 MB
- **Size of the generated dataset:** 1.18 MB
- **Total amount of disk used:** 15.85 MB
An example of 'train' looks as follows.
```
```
#### 18828_comp.sys.mac.hardware
- **Size of downloaded dataset files:** 14.67 MB
- **Size of the generated dataset:** 1.06 MB
- **Total amount of disk used:** 15.73 MB
An example of 'train' looks as follows.
```
```
### Data Fields
The data fields are the same among all splits.
#### 18828_alt.atheism
- `text`: a `string` feature.
#### 18828_comp.graphics
- `text`: a `string` feature.
#### 18828_comp.os.ms-windows.misc
- `text`: a `string` feature.
#### 18828_comp.sys.ibm.pc.hardware
- `text`: a `string` feature.
#### 18828_comp.sys.mac.hardware
- `text`: a `string` feature.
### Data Splits
| name |train|
|------------------------------|----:|
|18828_alt.atheism | 799|
|18828_comp.graphics | 973|
|18828_comp.os.ms-windows.misc | 985|
|18828_comp.sys.ibm.pc.hardware| 982|
|18828_comp.sys.mac.hardware | 961|
## Dataset Creation
### Curation Rationale
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Source Data
#### Initial Data Collection and Normalization
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### Who are the source language producers?
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Annotations
#### Annotation process
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### Who are the annotators?
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Personal and Sensitive Information
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## Considerations for Using the Data
### Social Impact of Dataset
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Discussion of Biases
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Other Known Limitations
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## Additional Information
### Dataset Curators
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Licensing Information
[More Information Needed](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### Citation Information
```
@incollection{LANG1995331,
title = {NewsWeeder: Learning to Filter Netnews},
editor = {Armand Prieditis and Stuart Russell},
booktitle = {Machine Learning Proceedings 1995},
publisher = {Morgan Kaufmann},
address = {San Francisco (CA)},
pages = {331-339},
year = {1995},
isbn = {978-1-55860-377-6},
doi = {https://doi.org/10.1016/B978-1-55860-377-6.50048-7},
url = {https://www.sciencedirect.com/science/article/pii/B9781558603776500487},
author = {Ken Lang},
}
```
### Contributions
Thanks to [@mariamabarham](https://github.com/mariamabarham), [@thomwolf](https://github.com/thomwolf), [@lhoestq](https://github.com/lhoestq) for adding this dataset.
# 「20新闻组（20 Newsgroups）」数据集卡片
## 目录
- [数据集描述](#dataset-description)
- [数据集概要](#dataset-summary)
- [支持任务与排行榜](#supported-tasks-and-leaderboards)
- [语言](#languages)
- [数据集结构](#dataset-structure)
- [数据实例](#data-instances)
- [数据字段](#data-fields)
- [数据拆分](#data-splits)
- [数据集构建](#dataset-creation)
- [构建依据](#curation-rationale)
- [源数据](#source-data)
- [标注信息](#annotations)
- [个人与敏感信息](#personal-and-sensitive-information)
- [数据集使用注意事项](#considerations-for-using-the-data)
- [数据集的社会影响](#social-impact-of-dataset)
- [偏差讨论](#discussion-of-biases)
- [其他已知局限性](#other-known-limitations)
- [附加信息](#additional-information)
- [数据集维护者](#dataset-curators)
- [许可信息](#licensing-information)
- [引用信息](#citation-information)
- [贡献致谢](#contributions)
## 数据集描述
- **主页：** [http://qwone.com/~jason/20Newsgroups/](http://qwone.com/~jason/20Newsgroups/)
- **代码仓库：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **相关论文：** [NewsWeeder: Learning to Filter Netnews](https://doi.org/10.1016/B978-1-55860-377-6.50048-7)
- **联系方式：** [更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
- **下载数据集文件大小：** 929.27 MB
- **生成后数据集大小：** 124.41 MB
- **总磁盘占用空间：** 1.05 GB
### 数据集概要
20新闻组（20 Newsgroups）数据集是约20000篇新闻组文档的集合，在20个不同的新闻组中近乎均匀划分。据目前所知，该数据集最初由肯·兰（Ken Lang）收集，大概率是为其论文《NewsWeeder：学习筛选网络新闻》（NewsWeeder: Learning to Filter Netnews）所筹备，尽管他并未在文中明确提及该数据集集合。20新闻组数据集现已成为机器学习技术文本应用（如文本分类、文本聚类）实验的热门数据集。
本数据集不含跨组交叉发布内容，仅保留「发件人（From）」与「主题（Subject）」头部字段。
### 支持任务与排行榜
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 语言
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## 数据集结构
### 数据实例
#### 18828_alt.atheism
- **下载数据集文件大小：** 14.67 MB
- **生成后数据集大小：** 1.67 MB
- **总磁盘占用空间：** 16.34 MB
「训练集」示例如下：
#### 18828_comp.graphics
- **下载数据集文件大小：** 14.67 MB
- **生成后数据集大小：** 1.66 MB
- **总磁盘占用空间：** 16.33 MB
「训练集」示例如下：
#### 18828_comp.os.ms-windows.misc
- **下载数据集文件大小：** 14.67 MB
- **生成后数据集大小：** 2.38 MB
- **总磁盘占用空间：** 17.05 MB
「训练集」示例如下：
#### 18828_comp.sys.ibm.pc.hardware
- **下载数据集文件大小：** 14.67 MB
- **生成后数据集大小：** 1.18 MB
- **总磁盘占用空间：** 15.85 MB
「训练集」示例如下：
#### 18828_comp.sys.mac.hardware
- **下载数据集文件大小：** 14.67 MB
- **生成后数据集大小：** 1.06 MB
- **总磁盘占用空间：** 15.73 MB
「训练集」示例如下：
### 数据字段
所有数据拆分的字段均保持一致。
#### 18828_alt.atheism
- `text`：字符串（string）类型特征。
#### 18828_comp.graphics
- `text`：字符串（string）类型特征。
#### 18828_comp.os.ms-windows.misc
- `text`：字符串（string）类型特征。
#### 18828_comp.sys.ibm.pc.hardware
- `text`：字符串（string）类型特征。
#### 18828_comp.sys.mac.hardware
- `text`：字符串（string）类型特征。
### 数据拆分
| 数据集子组 | 训练集样本数 |
|----------------------------------|-------------:|
|18828_alt.atheism | 799|
|18828_comp.graphics | 973|
|18828_comp.os.ms-windows.misc | 985|
|18828_comp.sys.ibm.pc.hardware | 982|
|18828_comp.sys.mac.hardware | 961|
## 数据集构建
### 构建依据
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 源数据
#### 初始数据收集与归一化
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### 源语言生成者是谁？
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 标注信息
#### 标注流程
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
#### 标注人员是谁？
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 个人与敏感信息
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## 数据集使用注意事项
### 数据集的社会影响
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 偏差讨论
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 其他已知局限性
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
## 附加信息
### 数据集维护者
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 许可信息
[更多信息待补充](https://github.com/huggingface/datasets/blob/master/CONTRIBUTING.md#how-to-contribute-to-the-dataset-cards)
### 引用信息
@incollection{LANG1995331,
title = {NewsWeeder: Learning to Filter Netnews},
editor = {Armand Prieditis and Stuart Russell},
booktitle = {Machine Learning Proceedings 1995},
publisher = {Morgan Kaufmann},
address = {San Francisco (CA)},
pages = {331-339},
year = {1995},
isbn = {978-1-55860-377-6},
doi = {https://doi.org/10.1016/B978-1-55860-377-6.50048-7},
url = {https://www.sciencedirect.com/science/article/pii/B9781558603776500487},
author = {Ken Lang},
}
### 贡献致谢
感谢[@mariamabarham](https://github.com/mariamabarham)、[@thomwolf](https://github.com/thomwolf)、[@lhoestq](https://github.com/lhoestq)为本数据集的收录提供支持。
提供机构：
maas
创建时间：
2025-07-07
