somewheresystems/dataclysm-pubmed

---
license: apache-2.0
language:
- en

tags:
- pubmed
- medical
- medicine
- NIH
- science
pretty_name: dataclysm-arxiv
size_categories:
- 10M<n<100M
---

# DATACLYSM PATCH 0.0.4: PUBMED
## USE THE NOTEBOOK TO GET STARTED!
https://github.com/somewheresystems/dataclysm

# somewheresystems/dataclysm-pubmed

This dataset comprises of 35.7 million PubMed metadata entries including title and some (~69% with) abstracts, with two new columns added: title-embeddings and abstract-embeddings. These additional columns were generated using the bge-small-en-v1.5 embeddings model. The dataset was sourced from the PubMed Baseline as of December 12, 2023. https://ftp.ncbi.nlm.nih.gov/pubmed/baseline/

# Embeddings Model

We used https://huggingface.co/BAAI/bge-small-en-v1.5 to embed the `title` and `abstract` fields.

## Contact

Please contact [e-mail] for inquiries.

原始信息汇总

数据集概述

数据集名称

• 名称: dataclysm-arxiv

数据集描述

• 内容: 包含3570万条PubMed元数据条目，包括标题和部分（约69%包含）摘要。新增两列：title-embeddings和abstract-embeddings。
• 来源: 数据来源于PubMed Baseline，截至2023年12月12日。

数据集特征

• 新增列:
  • title-embeddings
  • abstract-embeddings
• 嵌入模型: 使用bge-small-en-v1.5模型生成标题和摘要的嵌入。

数据集标签

• 标签:
  • pubmed
  • medical
  • medicine
  • NIH
  • science

数据集大小

• 大小: 10M<n<100M

语言

• 语言:
  • en

许可证

• 许可证: apache-2.0