bevaya/pubmed-ocr

--- language: - en license: other size_categories: - 1M<n<10M task_categories: - image-to-text - image-text-to-text pretty_name: PubMed-OCR arxiv: 2601.11425 dataset_info: features: - name: basename dtype: string - name: page dtype: int32 - name: license dtype: string - name: pmid dtype: string - name: accession_id dtype: string - name: article_citation dtype: string - name: pdf_bytes dtype: binary - name: ocr_json dtype: string configs: - config_name: default data_files: - split: train path: train-*.parquet license_name: pubmed-ocr-multiple-cc-licenses tags: - biology - medical - ocr - multimodal --- # PubMed-OCR: PMC Open Access OCR Annotations PubMed-OCR is an OCR-centric corpus of scientific articles derived from PubMed Central Open Access PDFs. Each **page** is rendered to an image and annotated with **Google Cloud Vision OCR**, released in a compact JSON schema with **word-, line-, and paragraph-level** bounding boxes. **Scale (release):** - **209.5K** articles - **~1.5M** pages - **~1.3B** words (OCR tokens) This dataset is intended to support layout-aware modeling, coordinate-grounded QA, and evaluation of OCR-dependent pipelines on scientific documents. ## Dataset Details ### Dataset Description - **Curated by:** Roots.ai - **Point of contact:** [e-mail] - **Language:** English (primarily; see limitations) - **Data unit:** **1 row = 1 PDF page** (unique by `{basename, page}`) - **License:** See **Licensing** section (source-article licenses; per-row `license` field) ### Dataset Sources - **Repository:** https://huggingface.co/datasets/rootsautomation/pubmed-ocr - **Paper:** [PubMed-OCR: PMC Open Access OCR Annotations](https://huggingface.co/papers/2601.11425) - **Source corpus:** PubMed Central Open Access (PMCOA) ## Uses ### Direct Use PubMed-OCR is suitable for: - Training/evaluating **OCR-aware** or **layout-aware** document models - Testing robustness of pipelines that depend on OCR (parsing, retrieval, extraction) - Building tasks that require **coordinate-grounded evidence** (e.g., quote-and-locate, region attribution) - Benchmark curation for scientific PDFs (tables, formulas, captions, references) ### Out-of-Scope Use - Do **not** treat OCR output as gold text; it contains recognition errors. - Not intended for clinical/medical decision-making. - Not intended for learning copyrighted content outside the applicable license terms. - Not intended as a reading-order ground truth dataset. ## Dataset Structure ### Data Instances Each row corresponds to a single page. Key identifiers: - `basename`: page group identifier (article-level) - `page`: page index within the article `ocr_json` is a JSON string containing OCR outputs with bounding boxes in **pixel coordinates** for the rendered page image. Example (schema sketch; fields may include additional metadata): ```json { "image": {"width": 1275, "height": 1650, "dpi": 150}, "text": { "words": [{"text": "Introduction", "bbox": [74, 132, 210, 156]}], "lines": [{"text": "Introduction", "bbox": [74, 130, 612, 160]}], "paragraphs": [{"text": "…", "bbox": [70, 120, 1180, 420]}] } } ``` ### Data Fields * `basename` *(string)*: article/page group identifier. * `page` *(int32)*: page index within the PDF/article. * `license` *(string)*: the **source article’s license** (e.g., `cc-by-4.0`, `cc-by-nc-4.0`, …). * `pmid` *(string)*: PubMed ID when available. * `accession_id` *(string)*: accession identifier (e.g., PMCID or internal ID). * `article_citation` *(string)*: a citation string for the source article. * `pdf_bytes` *(binary)*: raw PDF bytes **when redistribution is permitted**; may be empty/null otherwise. * `ocr_json` *(string)*: OCR output JSON (see above). ### Splits This release is provided as a single split (`train`) because it is primarily a **corpus**. For benchmarking, consider constructing evaluation splits that reduce leakage, e.g.: * **Journal-level splits** (hold out entire journals) * **Time-based splits** (hold out by publication year) * **PMID/PMCID disjoint splits** (article-level separation) ## Dataset Creation ### Curation Rationale Scientific PDFs are dense (formulas, tables, multi-column layouts). Many PMCOA datasets rely on PDF/XML alignment, which can miss scanned pages or inherit parser noise. PubMed-OCR provides OCR-native supervision directly from rendered page images, enabling OCR-dependent evaluation and layout-aware learning without PDF/XML alignment. ### Source Data #### Data Collection and Processing High-level pipeline: 1. Download PubMed Central Open Access PDFs (PMCOA) and filter to licenses permitting redistribution of derived artifacts. 2. Uniformly sample 209.5K documents. 3. Render each page at **150 DPI**. 4. Run **Google Cloud Vision** `document_text_detection` on page images. 5. Extract word- and paragraph-level polygons and canonicalize to axis-aligned bboxes `[x1, y1, x3, y3]`. 6. Reconstruct **line** bboxes by clustering words with similar vertical alignment (heuristic). 7. Emit one row per page with `ocr_json` (+ `pdf_bytes` where permitted). #### Who are the source data producers? The source texts were authored by scientific article authors and published via journals hosted in PubMed Central Open Access. ### Annotations #### Annotation process Annotations are machine-generated via Google Cloud Vision OCR. * **Words / paragraphs:** provided by the OCR engine * **Lines:** reconstructed heuristically from word boxes (see Limitations) #### Who are the annotators? The OCR engine is the annotator. No manual annotation was performed in this release. #### Personal and Sensitive Information Scientific articles can contain author names, affiliations, acknowledgements, emails, and citations. Content is drawn from publicly available PMCOA articles; no additional anonymization is applied. ## Bias, Risks, and Limitations * **Single OCR engine:** outputs reflect Google Vision’s strengths/weaknesses and may not generalize to other OCR systems. * **Heuristic line reconstruction:** line grouping and reading order can be imperfect, especially in multi-column layouts and around formulas/tables. * **Axis-aligned boxes:** original OCR polygons are simplified to rectangles. * **Domain skew:** PMCOA’s journal distribution is heavy-tailed (high-volume journals dominate). * **Non-text regions:** this dataset does not provide gold structure for tables/figures/formulas (only what OCR emits + derived lines). ### Recommendations * When reporting results, specify whether you use **words**, **lines**, or **paragraphs**, and whether you re-linearize text. * For fair evaluation, prefer **journal-disjoint** or **article-disjoint** splits. * If you need table/figure structure, pair this with a layout/table dataset (or run a layout model on top). ## Licensing This dataset contains content derived from PMCOA articles. * Each example inherits the **license of its source article**, recorded in the `license` field. * Users are responsible for complying with the license terms for any subset they use. * If `pdf_bytes` is present, it is provided only where redistribution is permitted. ## Citation If you use PubMed-OCR, please cite: ```bibtex @article{heidenreich2025pubmedocr, title={PubMed-OCR: PMC Open Access OCR Annotations}, author={Heidenreich, Hunter and Getachew, Yosheb and Dinica, Olivia and Elliott, Ben}, journal={arXiv preprint arXiv:2601.11425}, year={2025} } ``` ## How to Load ```python from datasets import load_dataset import json ds = load_dataset("rootsautomation/pubmed-ocr", split="train") row = ds[0] ocr = json.loads(row["ocr_json"]) words = ocr["text"]["words"] ``` For large-scale iteration, consider streaming: ```python ds = load_dataset("rootsautomation/pubmed-ocr", split="train", streaming=True) for row in ds: ocr = json.loads(row["ocr_json"]) ... ``` ## Models Trained on this Data - [GutenOCR-3B](https://huggingface.co/rootsautomation/GutenOCR-3B) - [GutenOCR-7B](https://huggingface.co/rootsautomation/GutenOCR-7B)--- 语言： - 英语 许可证：其他 规模类别： - 100万 < 样本量 < 1000万 任务类别： - 图像到文本 - 图像-文本到文本 正式名称：PubMed-OCR arXiv编号：2601.11425 数据集信息： 特征： - 字段名：basename，数据类型：字符串 - 字段名：page，数据类型：int32 - 字段名：license，数据类型：字符串 - 字段名：pmid，数据类型：字符串 - 字段名：accession_id，数据类型：字符串 - 字段名：article_citation，数据类型：字符串 - 字段名：pdf_bytes，数据类型：二进制 - 字段名：ocr_json，数据类型：字符串 配置： - 配置名称：default 数据文件： - 划分：训练集 路径：train-*.parquet 许可证名称：pubmed-ocr-multiple-cc-licenses 标签： - 生物学 - 医学 - OCR - 多模态 --- # PubMed-OCR：PubMed Central（PMC）开放获取OCR标注数据集 PubMed-OCR是一个以光学字符识别（Optical Character Recognition，以下简称OCR）为核心的科学文献语料库，源自PubMed Central（以下简称PMC）开放获取PDF文献。每一页文档均被渲染为图像，并通过**Google Cloud Vision OCR**进行标注，标注结果采用紧凑JSON格式存储，包含词级、行级与段落级的边界框（bounding box）。 **发布规模：** - **20.95万** 篇文献 - **约150万** 页文档 - **约13亿** 个单词（OCR Token） 本数据集旨在支持布局感知建模、基于坐标的问答（Question Answering，以下简称QA）任务，以及针对科学文献的OCR依赖型处理流水线的评估。 ## 数据集详情 ### 数据集描述 - **整理方：** Roots.ai - **联系人：** [e-mail] - **语言：** 以英语为主（详见局限性说明） - **数据单元：** **1行对应1个PDF页面**（由`{basename, page}`唯一标识） - **许可证：** 详见**许可证**章节（源文献许可证；每行数据的`license`字段记录对应许可证） ### 数据集来源 - **代码仓库：** https://huggingface.co/datasets/rootsautomation/pubmed-ocr - **相关论文：** [PubMed-OCR：PMC开放获取OCR标注数据集](https://huggingface.co/papers/2601.11425) - **源语料库：** PubMed Central开放获取（PMCOA） ## 数据集用途 ### 直接用途 PubMed-OCR适用于： - 训练/评估**OCR感知**或**布局感知**的文档模型 - 测试依赖OCR的处理流水线（包括解析、检索、提取任务）的鲁棒性 - 构建需要**坐标锚定证据**的任务（例如，引文定位、区域归因） - 为科学PDF文献构建基准测试集（涵盖表格、公式、图表标题、参考文献等场景） ### 不适用场景 - 请勿将OCR输出视为标准文本，其存在识别误差 - 不可用于临床或医疗决策 - 不可用于学习超出适用许可证条款范围的受版权保护内容 - 不可作为阅读顺序标注基准数据集 ## 数据集结构 ### 数据实例 每行对应单个页面，核心标识符包括： - `basename`：页面组标识符（文献级） - `page`：文献内的页面索引 `ocr_json`为JSON格式字符串，包含渲染后的页面图像的OCR输出结果，其中边界框采用**像素坐标**。 示例（架构概览，字段可能包含额外元数据）： json { "image": {"width": 1275, "height": 1650, "dpi": 150}, "text": { "words": [{"text": "引言", "bbox": [74, 132, 210, 156]}], "lines": [{"text": "引言", "bbox": [74, 130, 612, 160]}], "paragraphs": [{"text": "…", "bbox": [70, 120, 1180, 420]}] } } ### 数据字段 * `basename`（字符串类型）：文献/页面组标识符 * `page`（int32类型）：PDF/文献内的页面索引 * `license`（字符串类型）：**源文献的许可证**（例如`cc-by-4.0`、`cc-by-nc-4.0`等） * `pmid`（字符串类型）：可用时的PubMed编号 * `accession_id`（字符串类型）：登录标识符（例如PMCID或内部编号） * `article_citation`（字符串类型）：源文献的引用字符串 * `pdf_bytes`（二进制类型）：**允许再分发时**的原始PDF字节流；否则可能为空或null * `ocr_json`（字符串类型）：OCR输出JSON内容（详见上文） ### 数据划分 本次发布仅提供单一划分（`train`，训练集），因为本数据集本质为语料库。若需构建基准测试，建议构造可降低数据泄露的评估划分方式，例如： - **期刊级划分**：保留完整期刊作为测试集 - **时间级划分**：按发表年份划分测试集 - **PMID/PMCID互斥划分**：按文献级别进行分离，避免重叠 ## 数据集构建 ### 构建初衷 科学PDF文献内容紧凑（包含公式、表格、多栏布局等）。多数PMCOA数据集依赖PDF与XML的对齐，这可能会遗漏扫描页面或引入解析噪声。PubMed-OCR直接从渲染后的页面图像中获取OCR原生监督信号，无需依赖PDF/XML对齐即可支持OCR依赖型评估与布局感知学习。 ### 源数据 #### 数据收集与处理流程 核心处理流程如下： 1. 下载PubMed Central开放获取PDF文献（PMCOA），并筛选出允许再分发衍生产物的许可证类型的文献 2. 均匀采样20.95万篇文献 3. 将每一页以**150 DPI**分辨率渲染为图像 4. 对页面图像调用**Google Cloud Vision**的`document_text_detection`接口 5. 提取词级与段落级多边形边界框，并标准化为轴对齐的边界框`[x1, y1, x3, y3]` 6. 通过聚类垂直对齐相似的单词，启发式重构**行级**边界框 7. 为每个页面生成一行数据，包含`ocr_json`（允许时额外包含`pdf_bytes`） #### 源数据生产者 源文本由科学文献作者撰写，并通过PubMed Central开放获取平台的期刊发表。 ### 标注信息 #### 标注流程 标注结果由Google Cloud Vision OCR自动生成： - **单词/段落：** 由OCR引擎直接输出 - **行级框：** 由单词框启发式重构而来（详见局限性说明） #### 标注者 本发布版本的标注者为OCR引擎，未进行人工标注。 #### 个人与敏感信息 科学文献可能包含作者姓名、所属机构、致谢信息、邮箱地址与引用内容。本数据集内容来自公开可获取的PMCOA文献，未进行额外的匿名化处理。 ## 偏差、风险与局限性 * **单一OCR引擎：** 输出结果仅反映Google Vision的优缺点，无法泛化至其他OCR系统 * **启发式行框重构：** 行分组与阅读顺序可能存在误差，尤其在多栏布局以及公式/表格周围 * **轴对齐边界框：** 原始OCR多边形边界框被简化为矩形 * **领域偏差：** PMCOA的期刊分布呈重尾分布（高发文量期刊占主导） * **非文本区域：** 本数据集未提供表格、图表、公式的标准结构标注，仅包含OCR输出结果与重构的行框 ### 使用建议 * 报告实验结果时，请明确说明使用的是**单词级、行级还是段落级**标注，以及是否对文本进行了重线性化处理 * 为保证评估公平性，优先选择**期刊互斥划分**或**文献互斥划分** * 若需要表格/图表结构标注，建议结合布局/表格数据集使用（或在此基础上运行布局模型） ## 许可证说明 本数据集包含源自PMCOA文献的内容： - 每条数据继承其**源文献的许可证**，相关信息记录在`license`字段中 - 用户需对其使用的数据集子集遵守对应的许可证条款 - 仅在允许再分发的情况下，数据才会包含`pdf_bytes`字段 ## 引用方式 若使用PubMed-OCR数据集，请引用以下文献： bibtex @article{heidenreich2025pubmedocr, title={PubMed-OCR: PMC Open Access OCR Annotations}, author={Heidenreich, Hunter and Getachew, Yosheb and Dinica, Olivia and Elliott, Ben}, journal={arXiv预印本 arXiv:2601.11425}, year={2025} } ## 数据集加载方式 python from datasets import load_dataset import json ds = load_dataset("rootsautomation/pubmed-ocr", split="train") row = ds[0] ocr = json.loads(row["ocr_json"]) words = ocr["text"]["words"] 若需大规模迭代处理，可使用流式加载： python ds = load_dataset("rootsautomation/pubmed-ocr", split="train", streaming=True) for row in ds: ocr = json.loads(row["ocr_json"]) ... ## 基于本数据集训练的模型 - [GutenOCR-3B](https://huggingface.co/rootsautomation/GutenOCR-3B) - [GutenOCR-7B](https://huggingface.co/rootsautomation/GutenOCR-7B)