A Comprehensive Dataset of Classified Citations with Identifiers from English Wikipedia (2024)

2024 (new!) This is a dataset of 44.766.800 (+9.2%) citations extracted from the English Wikipedia February 2024 dump (https://dumps.wikimedia.org/enwiki/20240220/). The same extraction and template harmonization pipeline was used as the year before. The published dataset fields are like in the previous dataset. A classification label is assigned to each citation (either 'news', 'book', 'journal' or 'other)' by the deterministic rule-based classifier that analyses available identifiers (see code documentation for details), revealing the following citation subgroups: The total number of news: 10.958.151 (+9.4%) The total number of books:* 3.277.629 (+8.6%) The total number of journals*: 2.248.748 (+8.7%) * Please note that these numbers do not represent the overall number of book and journal citations, we count only citations with DOI, PMID, PMC and ISBN identifiers assigned by authors (prior to the lookup process that augments citations with missing identifiers). This dataset is not equipped with identifiers located via the lookup process (no 'acquired_ID_list' field). If there is interest in such an augmented version, see the source code for instructions or contact authors for assistance with this task. 2023 This is a dataset of 40.664.485 citations extracted from the English Wikipedia February 2023 dump (https://dumps.wikimedia.org/enwiki/20230220/). Version 1: en_citations.zip is a dataset of extracted citations Version 2: en_final.zip is the same dataset with classified citations augmented with identifiers The fields are as follows: type_of_citation - Wikipedia template type used to define the citation, e.g., 'cite journal', 'cite news', etc. page_title - title of the Wikipedia article from which the citation was extracted. Title - source title, e.g., title of the book, newspaper article, etc. URL - link to the source, e.g., webpage where news article was published, description of the book at the publisher's website, online library webpage, etc. tld - top link domain extracted from the URL, e.g., 'bbc' for https://www.bbc.co.uk/... Authors - list of article or book authors, if available. ID_list - list of publication identifiers mentioned in the citation, e.g., DOI, ISBN, etc. citations - citation text as used in Wikipedia code actual_label - 'book', 'journal', 'news', or 'other' label assigned based on the analysis of citation identifiers or top link domain. acquired_ID_list - identifiers located via Google Books and Crossref APIs for citations which are likely to refer to books or journals, i.e., defined using 'cite book', 'cite journal', 'cite encyclopedia', and 'cite proceedings' templates. The total number of news: 9.926.598 The total number of books: 2.994.601 The total number of journals: 2.052.172 Augmented with IDs via lookup 929.601 (out of 2.445.913 book, journal, encyclopedia, and proceedings template citations not classified as books or journals via given identifiers). The source code to extract citations can be found here: https://github.com/albatros13/wikicite. The code is a fork of the earlier project on Wikipedia citation extraction: https://github.com/Harshdeep1996/cite-classifications-wiki.2024（新版） 本数据集包含44,766,800条（+9.2%）引用，提取自2024年2月的英文维基百科数据转储（https://dumps.wikimedia.org/enwiki/20240220/）。 本次采用与上年完全一致的引用提取与模板标准化流程。发布的数据集字段与上一版数据集保持一致。每条引用均通过基于确定性规则的分类器完成标签赋值（标签可选「新闻」、「图书」、「期刊」或「其他」），该分类器通过分析可用的出版物标识符完成分类（详细规则参见代码文档），由此得到以下引用子类别的统计结果： 新闻类引用总量：10,958,151条（+9.4%） 图书类引用总量：3,277,629条（+8.6%）* 期刊类引用总量：2,248,748条（+8.7%）* *请注意：上述数值并非图书与期刊类引用的全部总量，本次统计仅包含作者已标注数字对象标识符（Digital Object Identifier, DOI）、PubMed ID（PMID）、PubMed Central ID（PMC）及国际标准书号（International Standard Book Number, ISBN）的引用（统计未包含后续通过补全流程为缺失标识符的引用补充标识的部分）。 本数据集未包含通过补全流程获取的标识符（无「acquired_ID_list」字段）。若需获取此类补全后的版本，请参阅源代码中的操作指南，或联系作者获取该任务的协助。 2023版 本数据集包含40,664,485条引用，提取自2023年2月的英文维基百科数据转储（https://dumps.wikimedia.org/enwiki/20230220/）。 版本1：en_citations.zip 为仅包含提取得到的引用的数据集 版本2：en_final.zip 为同一数据集，其中已为分类后的引用补充了标识符 数据集字段说明如下： - type_of_citation：用于定义引用的维基百科模板类型，例如「cite journal」、「cite news」等 - page_title：提取该引用的维基百科文章标题 - Title：来源文献标题，例如图书、新闻报道的标题等 - URL：来源链接，例如新闻报道的发布网页、出版社官网的图书介绍页面、在线图书馆网页等 - tld：从URL中提取的顶级链接域名，例如针对https://www.bbc.co.uk/... 的域名标识为「bbc」 - Authors：文献作者列表（如可获取） - ID_list：引用中提及的出版物标识符列表，例如DOI、ISBN等 - citations：维基百科代码中使用的引用原文 - actual_label：通过分析引用标识符或顶级链接域名赋值的标签，可选值为「book（图书）」、「journal（期刊）」、「news（新闻）」或「other（其他）」 - acquired_ID_list：通过Google Books与Crossref API获取的标识符，仅针对使用「cite book」、「cite journal」、「cite encyclopedia」及「cite proceedings」模板定义的、大概率指向图书或期刊的引用。 新闻类引用总量：9,926,598条 图书类引用总量：2,994,601条 期刊类引用总量：2,052,172条 通过补全流程为929,601条引用补充了标识符（该补全操作针对2,445,913条使用图书、期刊、百科全书及会议录模板定义，但无法通过现有标识符归类为图书或期刊的引用）。 引用提取的源代码可访问以下链接获取：https://github.com/albatros13/wikicite。 本代码为早期维基百科引用提取项目的分支复刻项目：https://github.com/Harshdeep1996/cite-classifications-wiki。