gebhart/patrepeval

PaTRepEval（版本v0.4）是一个用于评估专利训练嵌入模型的基准套件，相当于SciRepEval的专利版本。它基于美国专利商标局（USPTO）开放数据门户的授权专利和预授权公开表，并整合了外部数据源（如Kogan KPSS、Marx & Fuegi Reliance on Science）以支持Pool B/Phase 5任务。数据集包含多种配置，覆盖分类（如专利分类、受让人类型、CPC分类）、回归（如引用预测、专利价值）和检索（如专利与论文关联、相同发明人检索）任务。数据以parquet格式组织，包括训练集、测试集、语料库、查询和相关度判断文件，适用于嵌入模型的评估和基准测试。PaTRepEval — version v0.4 is a SciRepEval-equivalent evaluation suite for patent-trained embedding models. Built on USPTO Open Data Portal granted-patent and pre-grant publication tables, plus external sources (Kogan KPSS, Marx & Fuegi Reliance on Science) for the Pool B / Phase 5 tasks. It includes multiple configs for classification (e.g., patent classification, assignee type, CPC sections), regression (e.g., citation prediction, patent value), and retrieval (e.g., patent-paper linkage, same inventor retrieval) tasks. Data is stored in parquet format with splits such as train, test, corpus, queries, and qrels, designed for benchmarking embedding models.