Resources for the Evaluation of Automatic Indexing: Corpora and Gold Standard Indexes in the Agricultural, Educational, and Medical Domains

Three corpora have been constructed, each with its respective gold standard index, covering the domains of Agriculture, Education, and Medicine, for use in the evaluation of automatic indexing systems.The Agriculture corpus consists of four subcorpora. One contains 8,000 documents in plain text format, intended for training tasks, while the other three subcorpora contain 500 documents each, suitable for evaluation purposes. Each document in the Agriculture corpus includes a title, abstract, author keywords, and bibliographic references, metadata corresponding to scientific articles published in English in nine journals. The gold standard index for this domain was derived from the AGRICOLA database, maintained by the United States National Agricultural Library.The Education corpus is composed of four sets of documents. The main corpus contains 6,000 documents, and the other three sets include 450 documents each, all in plain text format. Each document in the Education corpus includes a title, abstract, author keywords, and bibliographic references, representing metadata from scientific articles published in English across more than 60 journals. The gold standard index for this domain comes from the ERIC database, provided by the Institute of Education Sciences (IES) of the U.S. Department of Education.The Medicine corpus is made up of five sets of documents. The main corpus includes 1,000 full-text scientific articles in XML-JATS format, published in Spanish across eight journals. The gold standard index for the medical domain originates from the LILACS database, maintained by BIREME in Brazil.本研究构建了三套语料库，每套均配有对应的金标准索引（gold standard index），覆盖农业、教育与医学三大领域，用于自动标引系统的评估。农业语料库包含四个子语料库：其中一个子语料库含8000篇纯文本格式文档，用于训练任务；其余三个子语料库各含500篇文档，适用于评估任务。农业语料库中的每篇文档均包含标题、摘要、作者关键词及参考文献，对应9种英文期刊发表的学术论文元数据。该领域的金标准索引源自美国国家农业图书馆维护的AGRICOLA数据库。教育语料库由四组文档构成：主语料库包含6000篇文档，其余三组各含450篇文档，全部为纯文本格式。教育语料库中的每篇文档均包含标题、摘要、作者关键词及参考文献，代表60余种英文期刊发表的学术论文元数据。该领域的金标准索引源自美国教育部教育科学研究所（Institute of Education Sciences, IES）提供的ERIC数据库。医学语料库包含五组文档：主语料库包含1000篇XML-JATS格式的全文学术论文，发表于8种西班牙语期刊。该领域的金标准索引源自巴西BIREME维护的LILACS数据库。