MA-tokenweights/all-the-news-2-tfidf-topic-stratified-v1-articles-sublinear

该数据集是一个用于自然语言处理的文本数据集，包含大量文章及其元数据。每篇文章具有唯一标识符（article_id）、文本内容（article_text）、单词列表（words）以及TF-IDF分数（tf_idf_scores）等特征。元数据包括标题（meta_title）、出版物来源（meta_publication）、发布日期（meta_date）、URL（meta_url）和章节（meta_section）。数据集还提供了分词信息，如令牌ID（token_ids）、令牌权重（token_weights）和子令牌标记（token_is_first_subtoken、token_is_last_subtoken），适用于文本分析、机器学习模型训练和评估任务。数据集分为训练集（train_articles）和验证集（validation_articles），分别包含153,426和929个示例。This dataset is a text dataset for natural language processing, containing numerous articles and their metadata. Each article includes a unique identifier (article_id), text content (article_text), word lists (words), and TF-IDF scores (tf_idf_scores), among other features. Metadata comprises title (meta_title), publication source (meta_publication), publication date (meta_date), URL (meta_url), and section (meta_section). The dataset also provides tokenization details, such as token IDs (token_ids), token weights (token_weights), and subtoken markers (token_is_first_subtoken, token_is_last_subtoken), making it suitable for text analysis, machine learning model training, and evaluation tasks. It is split into training (train_articles) and validation (validation_articles) sets, with 153,426 and 929 examples respectively.