Name: usersaico/modular-s2orc
Creator: usersaico
Published: 2026-03-16 19:27:17
License: 暂无描述

下载链接：

https://hf-mirror.com/datasets/usersaico/modular-s2orc

下载链接

链接失效反馈

官方服务：

资源简介：

--- license: odc-by language: - en tags: - text - science - evaluation - academic-documents - art - biology - chemistry - computer-science - medical - mathematics - physics - psychology - sociology size_categories: - 1M<n<10M dataset_info: - config_name: AgriculturalAndFoodSciences,1970-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43346 - name: validation num_examples: 106 - name: test num_examples: 90 - config_name: AgriculturalAndFoodSciences,2016-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 51765 - name: validation num_examples: 165 - name: test num_examples: 142 - config_name: AgriculturalAndFoodSciences,2019-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 88385 - name: validation num_examples: 230 - name: test num_examples: 268 - config_name: AgriculturalAndFoodSciences,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 56755 - name: validation num_examples: 153 - name: test num_examples: 155 - config_name: AgriculturalAndFoodSciences,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 32383 - name: validation num_examples: 107 - name: test num_examples: 118 - config_name: Art,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 29369 - name: validation num_examples: 83 - name: test num_examples: 84 - config_name: Biology,1970-1993 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 34380 - name: validation num_examples: 92 - name: test num_examples: 116 - config_name: Biology,1994-1999 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 30613 - name: validation num_examples: 79 - name: test num_examples: 74 - config_name: Biology,2000-2003 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 31667 - name: validation num_examples: 71 - name: test num_examples: 69 - config_name: Biology,2004-2006 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 34717 - name: validation num_examples: 90 - name: test num_examples: 117 - config_name: Biology,2007-2008 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 33939 - name: validation num_examples: 106 - name: test num_examples: 90 - config_name: Biology,2009-2010 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 56633 - name: validation num_examples: 161 - name: test num_examples: 152 - config_name: Biology,2011-2011 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 42178 - name: validation num_examples: 114 - name: test num_examples: 108 - config_name: Biology,2012-2012 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 54682 - name: validation num_examples: 124 - name: test num_examples: 144 - config_name: Biology,2013-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 64830 - name: validation num_examples: 178 - name: test num_examples: 194 - config_name: Biology,2014-2014 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 72827 - name: validation num_examples: 218 - name: test num_examples: 181 - config_name: Biology,2015-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 79102 - name: validation num_examples: 197 - name: test num_examples: 221 - config_name: Biology,2016-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 85961 - name: validation num_examples: 229 - name: test num_examples: 216 - config_name: Biology,2017-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 98250 - name: validation num_examples: 272 - name: test num_examples: 262 - config_name: Biology,2018-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 101178 - name: validation num_examples: 291 - name: test num_examples: 296 - config_name: Biology,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 110354 - name: validation num_examples: 322 - name: test num_examples: 309 - config_name: Biology,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 149329 - name: validation num_examples: 423 - name: test num_examples: 459 - config_name: Biology,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 164651 - name: validation num_examples: 484 - name: test num_examples: 500 - config_name: Biology,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 18380 - name: validation num_examples: 391 - name: test num_examples: 451 - config_name: Business,1970-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 32484 - name: validation num_examples: 82 - name: test num_examples: 82 - config_name: Business,2018-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 32818 - name: validation num_examples: 94 - name: test num_examples: 94 - config_name: Business,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 39443 - name: validation num_examples: 111 - name: test num_examples: 107 - config_name: Business,2021-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 59680 - name: validation num_examples: 188 - name: test num_examples: 192 - config_name: Chemistry,1970-2014 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40722 - name: validation num_examples: 124 - name: test num_examples: 108 - config_name: Chemistry,2015-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 42807 - name: validation num_examples: 120 - name: test num_examples: 129 - config_name: Chemistry,2019-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40396 - name: validation num_examples: 93 - name: test num_examples: 96 - config_name: Chemistry,2021-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 42413 - name: validation num_examples: 137 - name: test num_examples: 106 - config_name: ComputerScience,1970-2008 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 33431 - name: validation num_examples: 91 - name: test num_examples: 86 - config_name: ComputerScience,2009-2011 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 28912 - name: validation num_examples: 63 - name: test num_examples: 61 - config_name: ComputerScience,2012-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 31467 - name: validation num_examples: 79 - name: test num_examples: 75 - config_name: ComputerScience,2014-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43232 - name: validation num_examples: 121 - name: test num_examples: 106 - config_name: ComputerScience,2016-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 28402 - name: validation num_examples: 50 - name: test num_examples: 71 - config_name: ComputerScience,2017-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 35290 - name: validation num_examples: 101 - name: test num_examples: 77 - config_name: ComputerScience,2018-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48321 - name: validation num_examples: 131 - name: test num_examples: 133 - config_name: ComputerScience,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 61617 - name: validation num_examples: 166 - name: test num_examples: 185 - config_name: ComputerScience,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 110259 - name: validation num_examples: 301 - name: test num_examples: 317 - config_name: ComputerScience,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 136470 - name: validation num_examples: 344 - name: test num_examples: 354 - config_name: ComputerScience,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 110170 - name: validation num_examples: 304 - name: test num_examples: 326 - config_name: Economics,1970-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 27851 - name: validation num_examples: 70 - name: test num_examples: 60 - config_name: Economics,2017-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 37866 - name: validation num_examples: 113 - name: test num_examples: 100 - config_name: Economics,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 33363 - name: validation num_examples: 90 - name: test num_examples: 83 - config_name: Economics,2021-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48421 - name: validation num_examples: 140 - name: test num_examples: 137 - config_name: Education,1970-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 44679 - name: validation num_examples: 132 - name: test num_examples: 108 - config_name: Education,2019-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 65133 - name: validation num_examples: 174 - name: test num_examples: 156 - config_name: Education,2021-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 50667 - name: validation num_examples: 168 - name: test num_examples: 174 - config_name: Engineering,1970-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 39056 - name: validation num_examples: 88 - name: test num_examples: 120 - config_name: Engineering,2017-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 72614 - name: validation num_examples: 189 - name: test num_examples: 209 - config_name: Engineering,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 52607 - name: validation num_examples: 166 - name: test num_examples: 137 - config_name: Engineering,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 53806 - name: validation num_examples: 169 - name: test num_examples: 172 - config_name: Engineering,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 25494 - name: validation num_examples: 68 - name: test num_examples: 64 - config_name: EnvironmentalScience,1970-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 34552 - name: validation num_examples: 88 - name: test num_examples: 80 - config_name: EnvironmentalScience,2014-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 36710 - name: validation num_examples: 73 - name: test num_examples: 91 - config_name: EnvironmentalScience,2017-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40299 - name: validation num_examples: 113 - name: test num_examples: 109 - config_name: EnvironmentalScience,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 30827 - name: validation num_examples: 91 - name: test num_examples: 68 - config_name: EnvironmentalScience,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 52226 - name: validation num_examples: 140 - name: test num_examples: 156 - config_name: EnvironmentalScience,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 55234 - name: validation num_examples: 155 - name: test num_examples: 178 - config_name: EnvironmentalScience,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 28712 - name: validation num_examples: 89 - name: test num_examples: 102 - config_name: Geology,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 52252 - name: validation num_examples: 160 - name: test num_examples: 138 - config_name: History,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 16571 - name: validation num_examples: 32 - name: test num_examples: 51 - config_name: Linguistics,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 22730 - name: validation num_examples: 54 - name: test num_examples: 56 - config_name: MaterialsScience,1970-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 51178 - name: validation num_examples: 143 - name: test num_examples: 133 - config_name: MaterialsScience,2016-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 58515 - name: validation num_examples: 158 - name: test num_examples: 134 - config_name: MaterialsScience,2019-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 80015 - name: validation num_examples: 206 - name: test num_examples: 223 - config_name: MaterialsScience,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48883 - name: validation num_examples: 138 - name: test num_examples: 133 - config_name: MaterialsScience,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 30824 - name: validation num_examples: 99 - name: test num_examples: 88 - config_name: Mathematics,1970-2002 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 29186 - name: validation num_examples: 70 - name: test num_examples: 80 - config_name: Mathematics,2003-2006 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 27584 - name: validation num_examples: 87 - name: test num_examples: 64 - config_name: Mathematics,2007-2009 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 30061 - name: validation num_examples: 67 - name: test num_examples: 77 - config_name: Mathematics,2010-2011 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 28403 - name: validation num_examples: 68 - name: test num_examples: 72 - config_name: Mathematics,2012-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 36204 - name: validation num_examples: 78 - name: test num_examples: 83 - config_name: Mathematics,2014-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43291 - name: validation num_examples: 109 - name: test num_examples: 115 - config_name: Mathematics,2016-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 46835 - name: validation num_examples: 115 - name: test num_examples: 117 - config_name: Mathematics,2018-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 26529 - name: validation num_examples: 79 - name: test num_examples: 72 - config_name: Mathematics,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 30028 - name: validation num_examples: 70 - name: test num_examples: 96 - config_name: Mathematics,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43442 - name: validation num_examples: 124 - name: test num_examples: 133 - config_name: Mathematics,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48926 - name: validation num_examples: 119 - name: test num_examples: 137 - config_name: Mathematics,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40114 - name: validation num_examples: 90 - name: test num_examples: 98 - config_name: Medicine,1970-2006 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 50184 - name: validation num_examples: 130 - name: test num_examples: 139 - config_name: Medicine,2007-2009 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48764 - name: validation num_examples: 126 - name: test num_examples: 129 - config_name: Medicine,2010-2011 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 69379 - name: validation num_examples: 162 - name: test num_examples: 175 - config_name: Medicine,2012-2012 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 54263 - name: validation num_examples: 145 - name: test num_examples: 135 - config_name: Medicine,2013-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 68831 - name: validation num_examples: 150 - name: test num_examples: 158 - config_name: Medicine,2014-2014 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 84039 - name: validation num_examples: 225 - name: test num_examples: 220 - config_name: Medicine,2015-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 99661 - name: validation num_examples: 287 - name: test num_examples: 262 - config_name: Medicine,2016-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 114798 - name: validation num_examples: 292 - name: test num_examples: 316 - config_name: Medicine,2017-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 132983 - name: validation num_examples: 355 - name: test num_examples: 374 - config_name: Medicine,2018-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 144691 - name: validation num_examples: 424 - name: test num_examples: 372 - config_name: Medicine,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 168195 - name: validation num_examples: 456 - name: test num_examples: 445 - config_name: Medicine,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 276750 - name: validation num_examples: 748 - name: test num_examples: 839 - config_name: Medicine,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 294031 - name: validation num_examples: 910 - name: test num_examples: 884 - config_name: Medicine,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 222968 - name: validation num_examples: 674 - name: test num_examples: 739 - config_name: NA,1970-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: 'null' - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43379 - name: validation num_examples: 113 - name: test num_examples: 103 - config_name: NA,2020-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: 'null' - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 38294 - name: validation num_examples: 97 - name: test num_examples: 120 - config_name: Philosophy,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 16720 - name: validation num_examples: 50 - name: test num_examples: 54 - config_name: Physics,1970-1997 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40356 - name: validation num_examples: 103 - name: test num_examples: 117 - config_name: Physics,1998-2000 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 47324 - name: validation num_examples: 136 - name: test num_examples: 111 - config_name: Physics,2001-2002 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 38372 - name: validation num_examples: 88 - name: test num_examples: 96 - config_name: Physics,2003-2004 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43838 - name: validation num_examples: 103 - name: test num_examples: 110 - config_name: Physics,2005-2006 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 49373 - name: validation num_examples: 116 - name: test num_examples: 125 - config_name: Physics,2007-2008 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 57443 - name: validation num_examples: 170 - name: test num_examples: 162 - config_name: Physics,2009-2009 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 33644 - name: validation num_examples: 91 - name: test num_examples: 122 - config_name: Physics,2010-2010 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 35414 - name: validation num_examples: 94 - name: test num_examples: 86 - config_name: Physics,2011-2011 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 36926 - name: validation num_examples: 93 - name: test num_examples: 104 - config_name: Physics,2012-2012 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 39311 - name: validation num_examples: 102 - name: test num_examples: 103 - config_name: Physics,2013-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 41820 - name: validation num_examples: 121 - name: test num_examples: 94 - config_name: Physics,2014-2014 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 43698 - name: validation num_examples: 110 - name: test num_examples: 128 - config_name: Physics,2015-2015 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 45674 - name: validation num_examples: 104 - name: test num_examples: 126 - config_name: Physics,2016-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 48744 - name: validation num_examples: 126 - name: test num_examples: 140 - config_name: Physics,2017-2017 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 50248 - name: validation num_examples: 150 - name: test num_examples: 134 - config_name: Physics,2018-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 52807 - name: validation num_examples: 150 - name: test num_examples: 146 - config_name: Physics,2019-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 54381 - name: validation num_examples: 141 - name: test num_examples: 139 - config_name: Physics,2020-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 70544 - name: validation num_examples: 162 - name: test num_examples: 187 - config_name: Physics,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 78070 - name: validation num_examples: 217 - name: test num_examples: 225 - config_name: Physics,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 59784 - name: validation num_examples: 171 - name: test num_examples: 171 - config_name: PoliticalScience,1970-2019 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 28541 - name: validation num_examples: 74 - name: test num_examples: 98 - config_name: PoliticalScience,2020-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 36103 - name: validation num_examples: 106 - name: test num_examples: 111 - config_name: Psychology,1970-2013 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 36287 - name: validation num_examples: 85 - name: test num_examples: 89 - config_name: Psychology,2014-2016 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 42327 - name: validation num_examples: 111 - name: test num_examples: 119 - config_name: Psychology,2017-2018 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 38942 - name: validation num_examples: 89 - name: test num_examples: 106 - config_name: Psychology,2019-2020 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 58291 - name: validation num_examples: 176 - name: test num_examples: 183 - config_name: Psychology,2021-2021 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 40768 - name: validation num_examples: 131 - name: test num_examples: 147 - config_name: Psychology,2022-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 32538 - name: validation num_examples: 126 - name: test num_examples: 105 - config_name: Sociology,1970-2022 features: - name: added dtype: string - name: attributes struct: - name: bff_duplicate_paragraph_spans_decontamination sequence: 'null' - name: bff_contained_ngram_count dtype: int64 - name: bff_duplicate_spans sequence: 'null' - name: created dtype: string - name: id dtype: string - name: metadata struct: - name: extfieldsofstudy sequence: string - name: provenance dtype: string - name: s2fieldsofstudy sequence: string - name: sha1 dtype: string - name: year dtype: int64 - name: source dtype: string - name: text dtype: string - name: version dtype: string splits: - name: train num_examples: 32482 - name: validation num_examples: 81 - name: test num_examples: 101 configs: - config_name: AgriculturalAndFoodSciences,1970-2015 data_files: - split: train path: AgriculturalAndFoodSciences,1970-2015/train/* - split: validation path: AgriculturalAndFoodSciences,1970-2015/val/* - split: test path: AgriculturalAndFoodSciences,1970-2015/test/* - config_name: AgriculturalAndFoodSciences,2016-2018 data_files: - split: train path: AgriculturalAndFoodSciences,2016-2018/train/* - split: validation path: AgriculturalAndFoodSciences,2016-2018/val/* - split: test path: AgriculturalAndFoodSciences,2016-2018/test/* - config_name: AgriculturalAndFoodSciences,2019-2020 data_files: - split: train path: AgriculturalAndFoodSciences,2019-2020/train/* - split: validation path: AgriculturalAndFoodSciences,2019-2020/val/* - split: test path: AgriculturalAndFoodSciences,2019-2020/test/* - config_name: AgriculturalAndFoodSciences,2021-2021 data_files: - split: train path: AgriculturalAndFoodSciences,2021-2021/train/* - split: validation path: AgriculturalAndFoodSciences,2021-2021/val/* - split: test path: AgriculturalAndFoodSciences,2021-2021/test/* - config_name: AgriculturalAndFoodSciences,2022-2022 data_files: - split: train path: AgriculturalAndFoodSciences,2022-2022/train/* - split: validation path: AgriculturalAndFoodSciences,2022-2022/val/* - split: test path: AgriculturalAndFoodSciences,2022-2022/test/* - config_name: Art,1970-2022 data_files: - split: train path: Art,1970-2022/train/* - split: validation path: Art,1970-2022/val/* - split: test path: Art,1970-2022/test/* - config_name: Biology,1970-1993 data_files: - split: train path: Biology,1970-1993/train/* - split: validation path: Biology,1970-1993/val/* - split: test path: Biology,1970-1993/test/* - config_name: Biology,1994-1999 data_files: - split: train path: Biology,1994-1999/train/* - split: validation path: Biology,1994-1999/val/* - split: test path: Biology,1994-1999/test/* - config_name: Biology,2000-2003 data_files: - split: train path: Biology,2000-2003/train/* - split: validation path: Biology,2000-2003/val/* - split: test path: Biology,2000-2003/test/* - config_name: Biology,2004-2006 data_files: - split: train path: Biology,2004-2006/train/* - split: validation path: Biology,2004-2006/val/* - split: test path: Biology,2004-2006/test/* - config_name: Biology,2007-2008 data_files: - split: train path: Biology,2007-2008/train/* - split: validation path: Biology,2007-2008/val/* - split: test path: Biology,2007-2008/test/* - config_name: Biology,2009-2010 data_files: - split: train path: Biology,2009-2010/train/* - split: validation path: Biology,2009-2010/val/* - split: test path: Biology,2009-2010/test/* - config_name: Biology,2011-2011 data_files: - split: train path: Biology,2011-2011/train/* - split: validation path: Biology,2011-2011/val/* - split: test path: Biology,2011-2011/test/* - config_name: Biology,2012-2012 data_files: - split: train path: Biology,2012-2012/train/* - split: validation path: Biology,2012-2012/val/* - split: test path: Biology,2012-2012/test/* - config_name: Biology,2013-2013 data_files: - split: train path: Biology,2013-2013/train/* - split: validation path: Biology,2013-2013/val/* - split: test path: Biology,2013-2013/test/* - config_name: Biology,2014-2014 data_files: - split: train path: Biology,2014-2014/train/* - split: validation path: Biology,2014-2014/val/* - split: test path: Biology,2014-2014/test/* - config_name: Biology,2015-2015 data_files: - split: train path: Biology,2015-2015/train/* - split: validation path: Biology,2015-2015/val/* - split: test path: Biology,2015-2015/test/* - config_name: Biology,2016-2016 data_files: - split: train path: Biology,2016-2016/train/* - split: validation path: Biology,2016-2016/val/* - split: test path: Biology,2016-2016/test/* - config_name: Biology,2017-2017 data_files: - split: train path: Biology,2017-2017/train/* - split: validation path: Biology,2017-2017/val/* - split: test path: Biology,2017-2017/test/* - config_name: Biology,2018-2018 data_files: - split: train path: Biology,2018-2018/train/* - split: validation path: Biology,2018-2018/val/* - split: test path: Biology,2018-2018/test/* - config_name: Biology,2019-2019 data_files: - split: train path: Biology,2019-2019/train/* - split: validation path: Biology,2019-2019/val/* - split: test path: Biology,2019-2019/test/* - config_name: Biology,2020-2020 data_files: - split: train path: Biology,2020-2020/train/* - split: validation path: Biology,2020-2020/val/* - split: test path: Biology,2020-2020/test/* - config_name: Biology,2021-2021 data_files: - split: train path: Biology,2021-2021/train/* - split: validation path: Biology,2021-2021/val/* - split: test path: Biology,2021-2021/test/* - config_name: Biology,2022-2022 data_files: - split: train path: Biology,2022-2022/train/* - split: validation path: Biology,2022-2022/val/* - split: test path: Biology,2022-2022/test/* - config_name: Business,1970-2017 data_files: - split: train path: Business,1970-2017/train/* - split: validation path: Business,1970-2017/val/* - split: test path: Business,1970-2017/test/* - config_name: Business,2018-2019 data_files: - split: train path: Business,2018-2019/train/* - split: validation path: Business,2018-2019/val/* - split: test path: Business,2018-2019/test/* - config_name: Business,2020-2020 data_files: - split: train path: Business,2020-2020/train/* - split: validation path: Business,2020-2020/val/* - split: test path: Business,2020-2020/test/* - config_name: Business,2021-2022 data_files: - split: train path: Business,2021-2022/train/* - split: validation path: Business,2021-2022/val/* - split: test path: Business,2021-2022/test/* - config_name: Chemistry,1970-2014 data_files: - split: train path: Chemistry,1970-2014/train/* - split: validation path: Chemistry,1970-2014/val/* - split: test path: Chemistry,1970-2014/test/* - config_name: Chemistry,2015-2018 data_files: - split: train path: Chemistry,2015-2018/train/* - split: validation path: Chemistry,2015-2018/val/* - split: test path: Chemistry,2015-2018/test/* - config_name: Chemistry,2019-2020 data_files: - split: train path: Chemistry,2019-2020/train/* - split: validation path: Chemistry,2019-2020/val/* - split: test path: Chemistry,2019-2020/test/* - config_name: Chemistry,2021-2022 data_files: - split: train path: Chemistry,2021-2022/train/* - split: validation path: Chemistry,2021-2022/val/* - split: test path: Chemistry,2021-2022/test/* - config_name: ComputerScience,1970-2008 data_files: - split: train path: ComputerScience,1970-2008/train/* - split: validation path: ComputerScience,1970-2008/val/* - split: test path: ComputerScience,1970-2008/test/* - config_name: ComputerScience,2009-2011 data_files: - split: train path: ComputerScience,2009-2011/train/* - split: validation path: ComputerScience,2009-2011/val/* - split: test path: ComputerScience,2009-2011/test/* - config_name: ComputerScience,2012-2013 data_files: - split: train path: ComputerScience,2012-2013/train/* - split: validation path: ComputerScience,2012-2013/val/* - split: test path: ComputerScience,2012-2013/test/* - config_name: ComputerScience,2014-2015 data_files: - split: train path: ComputerScience,2014-2015/train/* - split: validation path: ComputerScience,2014-2015/val/* - split: test path: ComputerScience,2014-2015/test/* - config_name: ComputerScience,2016-2016 data_files: - split: train path: ComputerScience,2016-2016/train/* - split: validation path: ComputerScience,2016-2016/val/* - split: test path: ComputerScience,2016-2016/test/* - config_name: ComputerScience,2017-2017 data_files: - split: train path: ComputerScience,2017-2017/train/* - split: validation path: ComputerScience,2017-2017/val/* - split: test path: ComputerScience,2017-2017/test/* - config_name: ComputerScience,2018-2018 data_files: - split: train path: ComputerScience,2018-2018/train/* - split: validation path: ComputerScience,2018-2018/val/* - split: test path: ComputerScience,2018-2018/test/* - config_name: ComputerScience,2019-2019 data_files: - split: train path: ComputerScience,2019-2019/train/* - split: validation path: ComputerScience,2019-2019/val/* - split: test path: ComputerScience,2019-2019/test/* - config_name: ComputerScience,2020-2020 data_files: - split: train path: ComputerScience,2020-2020/train/* - split: validation path: ComputerScience,2020-2020/val/* - split: test path: ComputerScience,2020-2020/test/* - config_name: ComputerScience,2021-2021 data_files: - split: train path: ComputerScience,2021-2021/train/* - split: validation path: ComputerScience,2021-2021/val/* - split: test path: ComputerScience,2021-2021/test/* - config_name: ComputerScience,2022-2022 data_files: - split: train path: ComputerScience,2022-2022/train/* - split: validation path: ComputerScience,2022-2022/val/* - split: test path: ComputerScience,2022-2022/test/* - config_name: Economics,1970-2016 data_files: - split: train path: Economics,1970-2016/train/* - split: validation path: Economics,1970-2016/val/* - split: test path: Economics,1970-2016/test/* - config_name: Economics,2017-2019 data_files: - split: train path: Economics,2017-2019/train/* - split: validation path: Economics,2017-2019/val/* - split: test path: Economics,2017-2019/test/* - config_name: Economics,2020-2020 data_files: - split: train path: Economics,2020-2020/train/* - split: validation path: Economics,2020-2020/val/* - split: test path: Economics,2020-2020/test/* - config_name: Economics,2021-2022 data_files: - split: train path: Economics,2021-2022/train/* - split: validation path: Economics,2021-2022/val/* - split: test path: Economics,2021-2022/test/* - config_name: Education,1970-2018 data_files: - split: train path: Education,1970-2018/train/* - split: validation path: Education,1970-2018/val/* - split: test path: Education,1970-2018/test/* - config_name: Education,2019-2020 data_files: - split: train path: Education,2019-2020/train/* - split: validation path: Education,2019-2020/val/* - split: test path: Education,2019-2020/test/* - config_name: Education,2021-2022 data_files: - split: train path: Education,2021-2022/train/* - split: validation path: Education,2021-2022/val/* - split: test path: Education,2021-2022/test/* - config_name: Engineering,1970-2016 data_files: - split: train path: Engineering,1970-2016/train/* - split: validation path: Engineering,1970-2016/val/* - split: test path: Engineering,1970-2016/test/* - config_name: Engineering,2017-2019 data_files: - split: train path: Engineering,2017-2019/train/* - split: validation path: Engineering,2017-2019/val/* - split: test path: Engineering,2017-2019/test/* - config_name: Engineering,2020-2020 data_files: - split: train path: Engineering,2020-2020/train/* - split: validation path: Engineering,2020-2020/val/* - split: test path: Engineering,2020-2020/test/* - config_name: Engineering,2021-2021 data_files: - split: train path: Engineering,2021-2021/train/* - split: validation path: Engineering,2021-2021/val/* - split: test path: Engineering,2021-2021/test/* - config_name: Engineering,2022-2022 data_files: - split: train path: Engineering,2022-2022/train/* - split: validation path: Engineering,2022-2022/val/* - split: test path: Engineering,2022-2022/test/* - config_name: EnvironmentalScience,1970-2013 data_files: - split: train path: EnvironmentalScience,1970-2013/train/* - split: validation path: EnvironmentalScience,1970-2013/val/* - split: test path: EnvironmentalScience,1970-2013/test/* - config_name: EnvironmentalScience,2014-2016 data_files: - split: train path: EnvironmentalScience,2014-2016/train/* - split: validation path: EnvironmentalScience,2014-2016/val/* - split: test path: EnvironmentalScience,2014-2016/test/* - config_name: EnvironmentalScience,2017-2018 data_files: - split: train path: EnvironmentalScience,2017-2018/train/* - split: validation path: EnvironmentalScience,2017-2018/val/* - split: test path: EnvironmentalScience,2017-2018/test/* - config_name: EnvironmentalScience,2019-2019 data_files: - split: train path: EnvironmentalScience,2019-2019/train/* - split: validation path: EnvironmentalScience,2019-2019/val/* - split: test path: EnvironmentalScience,2019-2019/test/* - config_name: EnvironmentalScience,2020-2020 data_files: - split: train path: EnvironmentalScience,2020-2020/train/* - split: validation path: EnvironmentalScience,2020-2020/val/* - split: test path: EnvironmentalScience,2020-2020/test/* - config_name: EnvironmentalScience,2021-2021 data_files: - split: train path: EnvironmentalScience,2021-2021/train/* - split: validation path: EnvironmentalScience,2021-2021/val/* - split: test path: EnvironmentalScience,2021-2021/test/* - config_name: EnvironmentalScience,2022-2022 data_files: - split: train path: EnvironmentalScience,2022-2022/train/* - split: validation path: EnvironmentalScience,2022-2022/val/* - split: test path: EnvironmentalScience,2022-2022/test/* - config_name: Geology,1970-2022 data_files: - split: train path: Geology,1970-2022/train/* - split: validation path: Geology,1970-2022/val/* - split: test path: Geology,1970-2022/test/* - config_name: History,1970-2022 data_files: - split: train path: History,1970-2022/train/* - split: validation path: History,1970-2022/val/* - split: test path: History,1970-2022/test/* - config_name: Linguistics,1970-2022 data_files: - split: train path: Linguistics,1970-2022/train/* - split: validation path: Linguistics,1970-2022/val/* - split: test path: Linguistics,1970-2022/test/* - config_name: MaterialsScience,1970-2015 data_files: - split: train path: MaterialsScience,1970-2015/train/* - split: validation path: MaterialsScience,1970-2015/val/* - split: test path: MaterialsScience,1970-2015/test/* - config_name: MaterialsScience,2016-2018 data_files: - split: train path: MaterialsScience,2016-2018/train/* - split: validation path: MaterialsScience,2016-2018/val/* - split: test path: MaterialsScience,2016-2018/test/* - config_name: MaterialsScience,2019-2020 data_files: - split: train path: MaterialsScience,2019-2020/train/* - split: validation path: MaterialsScience,2019-2020/val/* - split: test path: MaterialsScience,2019-2020/test/* - config_name: MaterialsScience,2021-2021 data_files: - split: train path: MaterialsScience,2021-2021/train/* - split: validation path: MaterialsScience,2021-2021/val/* - split: test path: MaterialsScience,2021-2021/test/* - config_name: MaterialsScience,2022-2022 data_files: - split: train path: MaterialsScience,2022-2022/train/* - split: validation path: MaterialsScience,2022-2022/val/* - split: test path: MaterialsScience,2022-2022/test/* - config_name: Mathematics,1970-2002 data_files: - split: train path: Mathematics,1970-2002/train/* - split: validation path: Mathematics,1970-2002/val/* - split: test path: Mathematics,1970-2002/test/* - config_name: Mathematics,2003-2006 data_files: - split: train path: Mathematics,2003-2006/train/* - split: validation path: Mathematics,2003-2006/val/* - split: test path: Mathematics,2003-2006/test/* - config_name: Mathematics,2007-2009 data_files: - split: train path: Mathematics,2007-2009/train/* - split: validation path: Mathematics,2007-2009/val/* - split: test path: Mathematics,2007-2009/test/* - config_name: Mathematics,2010-2011 data_files: - split: train path: Mathematics,2010-2011/train/* - split: validation path: Mathematics,2010-2011/val/* - split: test path: Mathematics,2010-2011/test/* - config_name: Mathematics,2012-2013 data_files: - split: train path: Mathematics,2012-2013/train/* - split: validation path: Mathematics,2012-2013/val/* - split: test path: Mathematics,2012-2013/test/* - config_name: Mathematics,2014-2015 data_files: - split: train path: Mathematics,2014-2015/train/* - split: validation path: Mathematics,2014-2015/val/* - split: test path: Mathematics,2014-2015/test/* - config_name: Mathematics,2016-2017 data_files: - split: train path: Mathematics,2016-2017/train/* - split: validation path: Mathematics,2016-2017/val/* - split: test path: Mathematics,2016-2017/test/* - config_name: Mathematics,2018-2018 data_files: - split: train path: Mathematics,2018-2018/train/* - split: validation path: Mathematics,2018-2018/val/* - split: test path: Mathematics,2018-2018/test/* - config_name: Mathematics,2019-2019 data_files: - split: train path: Mathematics,2019-2019/train/* - split: validation path: Mathematics,2019-2019/val/* - split: test path: Mathematics,2019-2019/test/* - config_name: Mathematics,2020-2020 data_files: - split: train path: Mathematics,2020-2020/train/* - split: validation path: Mathematics,2020-2020/val/* - split: test path: Mathematics,2020-2020/test/* - config_name: Mathematics,2021-2021 data_files: - split: train path: Mathematics,2021-2021/train/* - split: validation path: Mathematics,2021-2021/val/* - split: test path: Mathematics,2021-2021/test/* - config_name: Mathematics,2022-2022 data_files: - split: train path: Mathematics,2022-2022/train/* - split: validation path: Mathematics,2022-2022/val/* - split: test path: Mathematics,2022-2022/test/* - config_name: Medicine,1970-2006 data_files: - split: train path: Medicine,1970-2006/train/* - split: validation path: Medicine,1970-2006/val/* - split: test path: Medicine,1970-2006/test/* - config_name: Medicine,2007-2009 data_files: - split: train path: Medicine,2007-2009/train/* - split: validation path: Medicine,2007-2009/val/* - split: test path: Medicine,2007-2009/test/* - config_name: Medicine,2010-2011 data_files: - split: train path: Medicine,2010-2011/train/* - split: validation path: Medicine,2010-2011/val/* - split: test path: Medicine,2010-2011/test/* - config_name: Medicine,2012-2012 data_files: - split: train path: Medicine,2012-2012/train/* - split: validation path: Medicine,2012-2012/val/* - split: test path: Medicine,2012-2012/test/* - config_name: Medicine,2013-2013 data_files: - split: train path: Medicine,2013-2013/train/* - split: validation path: Medicine,2013-2013/val/* - split: test path: Medicine,2013-2013/test/* - config_name: Medicine,2014-2014 data_files: - split: train path: Medicine,2014-2014/train/* - split: validation path: Medicine,2014-2014/val/* - split: test path: Medicine,2014-2014/test/* - config_name: Medicine,2015-2015 data_files: - split: train path: Medicine,2015-2015/train/* - split: validation path: Medicine,2015-2015/val/* - split: test path: Medicine,2015-2015/test/* - config_name: Medicine,2016-2016 data_files: - split: train path: Medicine,2016-2016/train/* - split: validation path: Medicine,2016-2016/val/* - split: test path: Medicine,2016-2016/test/* - config_name: Medicine,2017-2017 data_files: - split: train path: Medicine,2017-2017/train/* - split: validation path: Medicine,2017-2017/val/* - split: test path: Medicine,2017-2017/test/* - config_name: Medicine,2018-2018 data_files: - split: train path: Medicine,2018-2018/train/* - split: validation path: Medicine,2018-2018/val/* - split: test path: Medicine,2018-2018/test/* - config_name: Medicine,2019-2019 data_files: - split: train path: Medicine,2019-2019/train/* - split: validation path: Medicine,2019-2019/val/* - split: test path: Medicine,2019-2019/test/* - config_name: Medicine,2020-2020 data_files: - split: train path: Medicine,2020-2020/train/* - split: validation path: Medicine,2020-2020/val/* - split: test path: Medicine,2020-2020/test/* - config_name: Medicine,2021-2021 data_files: - split: train path: Medicine,2021-2021/train/* - split: validation path: Medicine,2021-2021/val/* - split: test path: Medicine,2021-2021/test/* - config_name: Medicine,2022-2022 data_files: - split: train path: Medicine,2022-2022/train/* - split: validation path: Medicine,2022-2022/val/* - split: test path: Medicine,2022-2022/test/* - config_name: NA,1970-2019 data_files: - split: train path: NA,1970-2019/train/* - split: validation path: NA,1970-2019/val/* - split: test path: NA,1970-2019/test/* - config_name: NA,2020-2022 data_files: - split: train path: NA,2020-2022/train/* - split: validation path: NA,2020-2022/val/* - split: test path: NA,2020-2022/test/* - config_name: Philosophy,1970-2022 data_files: - split: train path: Philosophy,1970-2022/train/* - split: validation path: Philosophy,1970-2022/val/* - split: test path: Philosophy,1970-2022/test/* - config_name: Physics,1970-1997 data_files: - split: train path: Physics,1970-1997/train/* - split: validation path: Physics,1970-1997/val/* - split: test path: Physics,1970-1997/test/* - config_name: Physics,1998-2000 data_files: - split: train path: Physics,1998-2000/train/* - split: validation path: Physics,1998-2000/val/* - split: test path: Physics,1998-2000/test/* - config_name: Physics,2001-2002 data_files: - split: train path: Physics,2001-2002/train/* - split: validation path: Physics,2001-2002/val/* - split: test path: Physics,2001-2002/test/* - config_name: Physics,2003-2004 data_files: - split: train path: Physics,2003-2004/train/* - split: validation path: Physics,2003-2004/val/* - split: test path: Physics,2003-2004/test/* - config_name: Physics,2005-2006 data_files: - split: train path: Physics,2005-2006/train/* - split: validation path: Physics,2005-2006/val/* - split: test path: Physics,2005-2006/test/* - config_name: Physics,2007-2008 data_files: - split: train path: Physics,2007-2008/train/* - split: validation path: Physics,2007-2008/val/* - split: test path: Physics,2007-2008/test/* - config_name: Physics,2009-2009 data_files: - split: train path: Physics,2009-2009/train/* - split: validation path: Physics,2009-2009/val/* - split: test path: Physics,2009-2009/test/* - config_name: Physics,2010-2010 data_files: - split: train path: Physics,2010-2010/train/* - split: validation path: Physics,2010-2010/val/* - split: test path: Physics,2010-2010/test/* - config_name: Physics,2011-2011 data_files: - split: train path: Physics,2011-2011/train/* - split: validation path: Physics,2011-2011/val/* - split: test path: Physics,2011-2011/test/* - config_name: Physics,2012-2012 data_files: - split: train path: Physics,2012-2012/train/* - split: validation path: Physics,2012-2012/val/* - split: test path: Physics,2012-2012/test/* - config_name: Physics,2013-2013 data_files: - split: train path: Physics,2013-2013/train/* - split: validation path: Physics,2013-2013/val/* - split: test path: Physics,2013-2013/test/* - config_name: Physics,2014-2014 data_files: - split: train path: Physics,2014-2014/train/* - split: validation path: Physics,2014-2014/val/* - split: test path: Physics,2014-2014/test/* - config_name: Physics,2015-2015 data_files: - split: train path: Physics,2015-2015/train/* - split: validation path: Physics,2015-2015/val/* - split: test path: Physics,2015-2015/test/* - config_name: Physics,2016-2016 data_files: - split: train path: Physics,2016-2016/train/* - split: validation path: Physics,2016-2016/val/* - split: test path: Physics,2016-2016/test/* - config_name: Physics,2017-2017 data_files: - split: train path: Physics,2017-2017/train/* - split: validation path: Physics,2017-2017/val/* - split: test path: Physics,2017-2017/test/* - config_name: Physics,2018-2018 data_files: - split: train path: Physics,2018-2018/train/* - split: validation path: Physics,2018-2018/val/* - split: test path: Physics,2018-2018/test/* - config_name: Physics,2019-2019 data_files: - split: train path: Physics,2019-2019/train/* - split: validation path: Physics,2019-2019/val/* - split: test path: Physics,2019-2019/test/* - config_name: Physics,2020-2020 data_files: - split: train path: Physics,2020-2020/train/* - split: validation path: Physics,2020-2020/val/* - split: test path: Physics,2020-2020/test/* - config_name: Physics,2021-2021 data_files: - split: train path: Physics,2021-2021/train/* - split: validation path: Physics,2021-2021/val/* - split: test path: Physics,2021-2021/test/* - config_name: Physics,2022-2022 data_files: - split: train path: Physics,2022-2022/train/* - split: validation path: Physics,2022-2022/val/* - split: test path: Physics,2022-2022/test/* - config_name: PoliticalScience,1970-2019 data_files: - split: train path: PoliticalScience,1970-2019/train/* - split: validation path: PoliticalScience,1970-2019/val/* - split: test path: PoliticalScience,1970-2019/test/* - config_name: PoliticalScience,2020-2022 data_files: - split: train path: PoliticalScience,2020-2022/train/* - split: validation path: PoliticalScience,2020-2022/val/* - split: test path: PoliticalScience,2020-2022/test/* - config_name: Psychology,1970-2013 data_files: - split: train path: Psychology,1970-2013/train/* - split: validation path: Psychology,1970-2013/val/* - split: test path: Psychology,1970-2013/test/* - config_name: Psychology,2014-2016 data_files: - split: train path: Psychology,2014-2016/train/* - split: validation path: Psychology,2014-2016/val/* - split: test path: Psychology,2014-2016/test/* - config_name: Psychology,2017-2018 data_files: - split: train path: Psychology,2017-2018/train/* - split: validation path: Psychology,2017-2018/val/* - split: test path: Psychology,2017-2018/test/* - config_name: Psychology,2019-2020 data_files: - split: train path: Psychology,2019-2020/train/* - split: validation path: Psychology,2019-2020/val/* - split: test path: Psychology,2019-2020/test/* - config_name: Psychology,2021-2021 data_files: - split: train path: Psychology,2021-2021/train/* - split: validation path: Psychology,2021-2021/val/* - split: test path: Psychology,2021-2021/test/* - config_name: Psychology,2022-2022 data_files: - split: train path: Psychology,2022-2022/train/* - split: validation path: Psychology,2022-2022/val/* - split: test path: Psychology,2022-2022/test/* - config_name: Sociology,1970-2022 data_files: - split: train path: Sociology,1970-2022/train/* - split: validation path: Sociology,1970-2022/val/* - split: test path: Sociology,1970-2022/test/* --- # Dataset Card for Modular S2ORC  Topically and temporally partitioned data from [S2ORC](https://aclanthology.org/2020.acl-main.447/), used for continued pre-training experiments in "Scalable Data Ablation Approximations for Language Models through Modular Training and Merging" to be presented at EMNLP 2024. Validation and test splits are determined by 'sha1' values in metadata. More details to come. ## Dataset Details ### Dataset Description  - **Curated by:** [More Information Needed] - **Funded by [optional]:** [More Information Needed] - **Shared by [optional]:** [More Information Needed] - **Language(s) (NLP):** English - **License:** [More Information Needed] ### Dataset Sources [optional]  - **Repository:** https://github.com/clarana/ez-data-ablations - **Paper:** [Scalable Data Ablation Approximations for Language Models through Modular Training and Merging](https://arxiv.org/abs/2410.15661) - **Demo [optional]:** [More Information Needed] ## Uses  ### Direct Use  [More Information Needed] ### Out-of-Scope Use  [More Information Needed] ## Dataset Structure  [More Information Needed] ## Dataset Creation ### Curation Rationale  [More Information Needed] ### Source Data  This dataset is a re-partitioned version of S2ORC, a corpus of scientific documents available through SemanticScholar. #### Data Collection and Processing  [More Information Needed] #### Who are the source data producers?  [More Information Needed] #### Personal and Sensitive Information  [More Information Needed] ## Bias, Risks, and Limitations  [More Information Needed] ### Recommendations  [More Information Needed] ## Citation [optional]  **[[ArXiv]](https://arxiv.org/abs/2410.15661)**: Scalable Data Ablation Approximations for Language Models through Modular Training and Merging **BibTeX:** ``` @inproceedings{na2024scalable, author = {Clara Na and Ian Magnusson and Ananya Harsh Jha and Tom Sherborne and Emma Strubell and Jesse Dodge and Pradeep Dasigi}, title = {Scalable Data Ablation Approximations for Language Models through Modular Training and Merging}, booktitle = {The 2024 Conference on Empirical Methods in Natural Language Processing, year = {2024}, url = {https://openreview.net/forum?id=clE6jmcnm4}, } ``` **APA:** [More Information Needed] ## Dataset Card Contact Clara Na (csna@andrew.cmu.edu)

## 数据集元数据许可证：开放数据公共领域署名许可（Open Data Commons Attribution License, ODC-BY）语言：英语（English）标签：文本、科学、评估、学术文献、艺术、生物学、化学、计算机科学、医学、数学、物理学、心理学、社会学样本量范围：100万 < 样本量 < 1000万数据集包含多个按主题与时间分区的配置，每个配置的字段结构统一，具体如下：每个配置包含以下字段： 1. `added`：字符串类型，数据添加相关信息 2. `attributes`：结构体，包含`bff_duplicate_paragraph_spans_decontamination`（空序列，段落重复跨度去污染相关）、`bff_contained_ngram_count`（64位整数，包含的n元语法计数）、`bff_duplicate_spans`（空序列，重复跨度信息） 3. `created`：字符串类型，数据创建时间 4. `id`：字符串类型，数据唯一标识 5. `metadata`：结构体，包含`extfieldsofstudy`（字符串序列，外部研究领域）、`provenance`（字符串类型，数据来源）、`s2fieldsofstudy`（字符串序列，S2研究领域）、`sha1`（字符串类型，SHA-1哈希值）、`year`（64位整数，出版年份） 6. `source`：字符串类型，原始数据源 7. `text`：字符串类型，文本内容 8. `version`：字符串类型，数据版本每个配置均包含训练集、验证集、测试集三个分割集，样本数量因配置的主题与时间区间不同而有所差异。所有配置的数据文件路径遵循统一格式：训练集数据路径为`{配置名称}/train/*`，验证集为`{配置名称}/val/*`，测试集为`{配置名称}/test/*` --- # 模块化S2ORC数据集卡片本数据集源自[S2ORC](https://aclanthology.org/2020.acl-main.447/)，按主题与时间维度进行分区，用于发表于2024年经验方法自然语言处理会议（EMNLP 2024）的论文《通过模块化训练与合并实现语言模型的可扩展数据消融近似》中的持续预训练实验。验证集与测试集根据元数据中的`sha1`哈希值划分。更多细节将后续公布。 ## 数据集详情 ### 数据集描述 - **整理者**：[需补充更多信息] - **资助方（可选）**：[需补充更多信息] - **共享方（可选）**：[需补充更多信息] - **自然语言**：英语（English） - **许可证**：开放数据公共领域署名许可（Open Data Commons Attribution License, ODC-BY） ### 数据集来源（可选） - **代码仓库**：https://github.com/clarana/ez-data-ablations - **相关论文**：[通过模块化训练与合并实现语言模型的可扩展数据消融近似](https://arxiv.org/abs/2410.15661) - **演示Demo（可选）**：[需补充更多信息] ## 用途 ### 直接用途 [需补充更多信息] ### 超出范围的用途 [需补充更多信息] ## 数据集结构 [需补充更多信息] ## 数据集创建 ### 整理依据 [需补充更多信息] ### 源数据本数据集是S2ORC的重新分区版本，S2ORC是可通过Semantic Scholar获取的科学文献语料库。 #### 数据收集与处理 [需补充更多信息] #### 源数据生产者 [需补充更多信息] #### 个人与敏感信息请说明数据集是否包含可被视为个人、敏感或隐私的数据（例如：揭示地址、唯一可识别的姓名或别名、种族或族裔出身、性取向、宗教信仰、政治观点、财务或健康数据等）。若已对数据进行匿名化处理，请描述匿名化流程。 [需补充更多信息] ## 偏差、风险与局限性 [需补充更多信息] ### 建议 [需补充更多信息] ## 引用（可选） **[ArXiv链接](https://arxiv.org/abs/2410.15661)**：通过模块化训练与合并实现语言模型的可扩展数据消融近似 **BibTeX格式引用**： @inproceedings{na2024scalable, author = {Clara Na and Ian Magnusson and Ananya Harsh Jha and Tom Sherborne and Emma Strubell and Jesse Dodge and Pradeep Dasigi}, title = {Scalable Data Ablation Approximations for Language Models through Modular Training and Merging}, booktitle = {The 2024 Conference on Empirical Methods in Natural Language Processing, year = {2024}, url = {https://openreview.net/forum?id=clE6jmcnm4}, } **APA格式引用**： [需补充更多信息] ## 数据集卡片联系人 Clara Na (csna@andrew.cmu.edu)

应用场景：