Softcite software mention extraction from the CORD-19 publications

<strong>Softcite software mention extraction from the CORD-19 publications </strong> This dataset is the first result of the extraction of software mentions from the set of publications of the CORD-19 corpus (https://allenai.org/data/cord-19) by the Softcite software recognizer, see https://github.com/ourresearch/software-mentions. The CORD-19 version used for this dataset is the one dated <strong>2021-03-22,</strong> using the <em>metadata.csv</em> file only. We re-harvested the PDF with https://github.com/kermitt2/article-dataset-builder in order to also extract coordinates of software mentions in the PDF and to take advantage of the latest version of GROBID to produce better full text extraction from PDF. <strong>Data format </strong> The extraction consists of 3 JSON files: <strong>annotations.json</strong> contains the individual software annotations including <em>software name</em> and possible attached attributes (<em>publisher</em>, <em>URL</em> and <em>version</em>). Each annotation is associated with coordinates expressed as bounding boxes in the original PDF. See Coordinates of structures in the original PDF for more details on the coordinate format. The context of citation is the sentence where the software name and its attributes are extracted. It is added to the JSON structure (field <em>context</em>), as well as the identifier of the document where the annotation belongs (field <em>document</em>, pointing to entries available in <em>documents.json</em>) and a list of bibliographical references attached to the software name (field <em>references</em>, pointing to entries available in <em>references.json</em>, with the used reference marker string). See https://github.com/ourresearch/software-mentions for more details on the extracted attributes. If the software name was sucessfully disambiguated against WikiData ("entity linking"), it appears in the field <em>wikidataId</em> as Wikidata entity identifier and in the field <em>wikipediaExternalRef</em> as a Wikipedia PageID from the English Wikipedia. Entity linking is realized with entity-fishing. <strong>documents.json</strong> contains the metadata of the all the CORD-19 documents containing at least one software annotation. The metadata are given as a CrossRef JSON structure. The abstract should be included in the metadata most of the time, as well as some complements extracted by GROBID directly from the PDF. In addition, the size of the pages and the unique file path to the PDF can be found to allow annotations directly on the PDF (see Coordinates of structures in the original PDF for more details on the PDF annotation display mechanism). <strong>references.json </strong>contains the parsed reference entries associated to software mentions. These references are given in the field <em>tei</em> encoded in the XML TEI format of GROBID extraction. The extracted raw references have been matched against CrossRef to get a DOI and more complete metadata with biblio-glutton. <strong>Statistics</strong> CORD-19 version: 2021-03-22 - total Open Access full texts: 211,213<br> - with at least one software mention: 76,448 - total software name annotations: 318,138<br> - with linked Wikidata ID: 117,193 - associated field <br> - publisher: 62,240 <br> - version: 105,661<br> - URL: 29,753 - associated bibliographical references: 61,170<br> - distinct references: 15,931<br> - distinct with matched DOI: 10,611<br> - distinct with matched PMC ID: 6,435 <strong>License and acknowledgements</strong> This dataset is licensed under a Creative Commons Attribution 4.0 International License. We thank Alfred P. Sloan Foundation for supporting this work.<strong>从CORD-19出版物中提取软件提及的Softcite数据集</strong> 本数据集为通过Softcite软件识别器（Softcite software recognizer）从CORD-19语料库（https://allenai.org/data/cord-19）的出版物集中提取软件提及的首批成果，详情参见https://github.com/ourresearch/software-mentions。本数据集所使用的CORD-19版本为2021年3月22日发布的版本，且仅使用其中的<em>metadata.csv</em>文件。我们通过https://github.com/kermitt2/article-dataset-builder重新获取了PDF文件，以便同时提取PDF中软件提及的坐标，并借助最新版GROBID（GROBID）实现更优质的PDF全文提取。<strong>数据格式</strong> 本次提取共生成3个JSON格式文件：<strong>annotations.json</strong>包含单条软件标注信息，涵盖<em>软件名称</em>以及可附带的属性字段（<em>出版商</em>、<em>URL</em>和<em>版本号</em>）。每条标注均关联原始PDF中的边界框（bounding box）坐标，关于坐标格式的详细说明请参见《原始PDF中的结构坐标》文档。引用上下文为提取到软件名称及其属性的句子，该内容将被加入JSON结构的<em>context</em>字段中；同时还会包含标注所属文档的标识符（<em>document</em>字段，指向<em>documents.json</em>中的条目），以及依附于该软件名称的参考文献列表（<em>references</em>字段，指向<em>references.json</em>中的条目，并附带使用的参考文献标记字符串）。关于提取属性的更多详情，请参见https://github.com/ourresearch/software-mentions。若软件名称成功通过维基数据（WikiData）完成消歧（即“实体链接”），则其对应的维基数据实体标识符将存入<em>wikidataId</em>字段，英文维基百科的页面ID将存入<em>wikipediaExternalRef</em>字段。本次实体链接任务通过entity-fishing工具实现。<strong>documents.json</strong>包含所有至少含有一条软件标注的CORD-19文献的元数据，元数据采用CrossRef（交叉引用服务）的JSON格式。大多数情况下，元数据中会包含文献摘要，以及GROBID直接从PDF中提取的补充元数据。此外，文件还提供了PDF的页面尺寸与唯一文件路径，以便直接在PDF上进行标注（关于PDF标注显示机制的详细说明，请参见《原始PDF中的结构坐标》文档）。<strong>references.json</strong>包含与软件提及相关的解析后参考文献条目，这些参考文献以GROBID提取的XML TEI（文本编码倡议）格式存储在<em>tei</em>字段中。提取得到的原始参考文献已通过biblio-glutton工具与CrossRef进行匹配，以获取DOI（数字对象唯一标识符）及更完整的元数据。<strong>统计数据</strong> CORD-19版本：2021-03-22<br>- 开源访问全文总数：211,213篇<br>- 至少包含1条软件提及的文献：76,448篇<br>- 软件名称标注总数量：318,138条<br>- 关联维基数据标识符的标注：117,193条<br> - 关联出版商字段：62,240条<br> - 关联版本号字段：105,661条<br> - 关联URL字段：29,753条<br>- 关联参考文献的标注：61,170条<br>- 唯一参考文献数量：15,931条<br>- 匹配到DOI的唯一参考文献：10,611条<br>- 匹配到PMC ID（PubMed Central标识符）的唯一参考文献：6,435条<strong>授权与致谢</strong> 本数据集采用知识共享署名4.0国际通用许可协议（Creative Commons Attribution 4.0 International License）进行授权。感谢阿尔弗雷德·P·斯隆基金会（Alfred P. Sloan Foundation）对本项目的支持。