THUCTC 中文文本分类数据集

THUCTC(THU Chinese Text Classification)是由清华大学自然语言处理实验室推出的中文文本分类工具包，能够自动高效地实现用户自定义的文本分类语料的训练、评测、分类功能。文本分类通常包括特征选取、特征降维、分类模型学习三个步骤。如何选取合适的文本特征并进行降维，是中文文本分类的挑战性问题。我组根据多年在中文文本分类的研究经验，在THUCTC中选取二字串bigram作为特征单元，特征降维方法为Chi-square，权重计算方法为tfidf，分类模型使用的是LibSVM或LibLinear。THUCTC对于开放领域的长文本具有良好的普适性，不依赖于任何中文分词工具的性能，具有准确率高、测试速度快的优点。 我们提供了两种方式运行工具包： [-c CATEGORY_LIST_FILE_PATH] 从文件中读入类别信息。该文件中每行包含且仅包含一个类别名称。 [-train TRAIN_PATH] 进行训练，并设置训练语料文件夹路径。该文件夹下每个子文件夹的名称都对应一个类别名称，内含属于该类别的训练语料。若不设置，则不进行训练。 [-test eval_PATH] 进行评测，并设置评测语料文件夹路径。该文件夹下每个子文件夹的名称都对应一个类别名称，内含属于该类别的评测语料。若不设置，则不进行评测。也可以使用-eval。 [-classify FILE_PATH] 对一个文件进行分类。 [-n topN] 设置返回候选分类数，按得分大小排序。默认为1，即只返回最可能的分类。 [-svm libsvm or liblinear] 选择使用libsvm还是liblinear进行训练和测试，默认使用liblinear。 [-l LOAD_MODEL_PATH] 设置读取模型路径。 [-s SAVE_MODEL_PATH] 设置保存模型路径。 [-f FEATURE_SIZE] 设置保留特征数目，默认为5000。 [-d1 RATIO] 设置训练集占总文件数比例，默认为0.8。 [-d2 RATIO] 设置测试集占总文件数比例，默认为0.2。 [-e ENCODING] 设置训练及测试文件编码，默认为UTF-8。 [-filter SUFFIX] 设置文件后缀过滤。例如设置“-filter .txt”，则训练和测试时仅考虑文件名后缀为.txt的文件 我们随工具包提供了一个调用THUCTC的样例代码Demo.java，其中实现了三种功能： BasicTextClassifier 是系统的入口类,提供多种设置接口供使用者调用。利用此入口类可以从文件中读入别信息、设置训练语料路径、设置训练参数以及模型保存路径等。 其中常用的类成员函数包括： public void Init(String[] args) 功能：输入运行参数，初始化系统。 public void runAsBigramChineseTextClassifier() 功能：根据参数，运行系统。 public boolean loadCategoryListFromFile(String filePath) 功能：从文件中获取分类列表，等同于参数-c filePath public boolean loadCategoryListFromFolder(String folder) 功能：从文件夹中获取分类列表 public void addTrainingText(String category, String filename) 功能：给定类别，添加训练文本 public void addfiles(String filename) 功能：根据训练文件所在的文件夹名称,自动判别类别并加入训练，等同于参数-train filename public ClassifyResult[] classifyFile(String filepath, int topN) 功能：对一个文件进行分类，返回前 topN 个分类结果。如果输入的 filepath 是文件夹,则只会在 Console 中打印每个子文件的分类结果，返回值是空数组，等同于参数-classify filepath -n topN public ClassifyResult[] classifyText(String text, int topN) 功能：对一个文本进行分类,返回前 topN 个分类结果 public void testfiles(String filename) 功能：对文件进行自动分类测试，等同于参数-test filename public double getPrecision() 功能：获得测试准确率 THUCNews是根据新浪新闻RSS订阅频道2005~2011年间的历史数据筛选过滤生成，包含74万篇新闻文档（2.19 GB），均为UTF-8纯文本格式。我们在原始新浪新闻分类体系的基础上，重新整合划分出14个候选分类类别：财经、彩票、房产、股票、家居、教育、科技、社会、时尚、时政、体育、星座、游戏、娱乐。使用THUCTC工具包在此数据集上进行评测，准确率可以达到88.6%。 文本分类的性能评价有多种指标，其中主流的文本分类评价指标包括准确率、召回率、F-measure、微平均与宏平均等。其中，微平均指所有样本的测试结果的算数平均值，宏平均指所有类别的测试结果的算数平均值。我们的测试也主要对这些指标进行测试。 我们选取上节介绍的数据集进行测试，测试时使用以下参数组合(-d1 -d2),(-f): Maosong Sun（孙茂松，导师），Jingyang Li（李景阳，博士生），Zhipeng Guo（郭志芃，本科生），Yu Zhao（赵宇，博士生），Yabin Zheng（郑亚斌，博士生），Xiance Si（司宪策，博士生），Zhiyuan Liu（刘知远，助理教授）. 使用者如有任何问题、建议和意见，欢迎发邮件至 [e-mail] 。

THUCTC (THU Chinese Text Classification) is a Chinese text classification toolkit developed by the Natural Language Processing Laboratory of Tsinghua University, which can automatically and efficiently implement the training, evaluation and classification functions of user-defined text classification corpora. Text classification usually includes three steps: feature selection, feature dimensionality reduction and classification model learning. How to select appropriate text features and perform dimensionality reduction is a challenging problem in Chinese text classification. Based on years of research experience in Chinese text classification, our team adopts bigram as the feature unit in THUCTC, uses Chi-square as the feature dimensionality reduction method, tfidf as the weight calculation method, and uses LibSVM or LibLinear as the classification model. THUCTC has good generality for open-domain long texts, does not rely on the performance of any Chinese word segmentation tool, and has the advantages of high accuracy and fast testing speed.

We provide two ways to run the toolkit:
[-c CATEGORY_LIST_FILE_PATH]: Read category information from a file. Each line in this file contains exactly one category name.
[-train TRAIN_PATH]: Start training and set the path of the training corpus folder. The name of each subfolder under this folder corresponds to a category name, and the subfolder contains training corpus belonging to this category. If not set, no training will be performed.
[-test EVAL_PATH]: Start evaluation and set the path of the evaluation corpus folder. The name of each subfolder under this folder corresponds to a category name, and the subfolder contains evaluation corpus belonging to this category. If not set, no evaluation will be performed. Alternatively, use -eval.
[-classify FILE_PATH]: Classify a single file.
[-n topN]: Set the number of returned candidate classifications, sorted by score. The default value is 1, that is, only the most likely classification is returned.
[-svm libsvm or liblinear]: Select whether to use libsvm or liblinear for training and testing. The default is liblinear.
[-l LOAD_MODEL_PATH]: Set the path for loading the pre-trained model.
[-s SAVE_MODEL_PATH]: Set the path for saving the trained model.
[-f FEATURE_SIZE]: Set the number of reserved features, the default is 5000.
[-d1 RATIO]: Set the proportion of the training set in the total number of files, the default is 0.8.
[-d2 RATIO]: Set the proportion of the test set in the total number of files, the default is 0.2.
[-e ENCODING]: Set the encoding of training and test files, the default is UTF-8.
[-filter SUFFIX]: Set file suffix filtering. For example, if you set "-filter .txt", only files with the .txt suffix will be considered during training and testing.

We provide a sample code Demo.java for calling THUCTC along with the toolkit, which implements three functions:
BasicTextClassifier is the entry class of the system, providing multiple setting interfaces for users to call. Using this entry class, users can read category information from files, set training corpus paths, set training parameters and model saving paths, etc. Common member functions include:
- public void Init(String[] args): Function: Input running parameters to initialize the system.
- public void runAsBigramChineseTextClassifier(): Function: Run the system according to the parameters.
- public boolean loadCategoryListFromFile(String filePath): Function: Obtain the classification list from a file, which is equivalent to the parameter -c filePath.
- public boolean loadCategoryListFromFolder(String folder): Function: Obtain the classification list from a folder.
- public void addTrainingText(String category, String filename): Function: Given a category, add training text.
- public void addfiles(String filename): Function: Automatically judge the category and add training texts according to the folder name where the training files are located, which is equivalent to the parameter -train filename.
- public ClassifyResult[] classifyFile(String filepath, int topN): Function: Classify a single file and return the top N classification results. If the input filepath is a folder, only the classification results of each sub-file will be printed on the Console, and the return value is an empty array, which is equivalent to the parameter -classify filepath -n topN.
- public ClassifyResult[] classifyText(String text, int topN): Function: Classify a given text and return the top N classification results.
- public void testfiles(String filename): Function: Automatically perform classification test on files, which is equivalent to the parameter -test filename.
- public double getPrecision(): Function: Obtain the test accuracy.

THUCNews is generated by filtering and screening historical data from Sina News RSS subscription channels between 2005 and 2011, containing 740,000 news documents (2.19 GB) in pure UTF-8 text format. We reorganized and divided 14 candidate classification categories based on the original Sina News classification system: Finance, Lottery, Real Estate, Stocks, Home Furnishing, Education, Technology, Society, Fashion, Politics, Sports, Constellation, Games, and Entertainment. When using the THUCTC toolkit to evaluate on this dataset, the accuracy can reach 88.6%.

There are multiple indicators for evaluating text classification performance, among which the mainstream ones include accuracy, recall rate, F-measure, micro-average and macro-average. Micro-average refers to the arithmetic average of the test results of all samples, while macro-average refers to the arithmetic average of the test results of all categories. Our test mainly evaluates these indicators.

We selected the dataset introduced in the previous section for testing, and used the following parameter combinations during testing: (-d1 -d2), (-f).

Authors: Maosong Sun (Supervisor), Jingyang Li (PhD Student), Zhipeng Guo (Undergraduate Student), Yu Zhao (PhD Student), Yabin Zheng (PhD Student), Xiance Si (PhD Student), Zhiyuan Liu (Assistant Professor).

Users are welcome to send emails to [e-mail] for any questions, suggestions or comments.

搜集汇总

背景与挑战

背景概述

THUCTC中文文本分类数据集是一个由清华大学开发的高效文本分类工具包，包含74万篇新闻文档，覆盖14个类别，采用bigram特征和Chi-square降维方法，适用于开放领域的长文本分类，准确率高且测试速度快。