Molecular Biology Open Access Pubmed Word and Sentence Representations

<strong>Natural Language Embeddings about Molecular Biology</strong> This dataset is concerned with developing a tailored training data set for word and sentence embedding based on biomedical text that has some component associated with molecular work (as opposed to the other range of work indexed in PubMed like non molecular clinical work, studies of human behavior, etc). <strong>Raw Data</strong> In order to develop natural language embeddings (for words and sentences), we queried PMC and MEDLINE for molecular papers only by using high-level MeSH terms to restrict interest to papers with a molecular focus. We used the following MeSH terms: Cells [A11] Multiprotein Complexes [D05.500] Protein Aggregates [D05.875] Hormones [D06] Enzymes and Coenzymes [D08] Carbohydrates [D08] Lipids [D10] Amino Acids, Peptides and Proteins [D12] Nucleic Acids, Nucleotides and Nucleosides [D13] Biological Factors [D23] Pharmaceutical Preparations [D26] Metabolism [G03] Genetic Phenomena [G06] Queries for these terms use the following string: "cells"[MeSH Terms] OR "Multiprotein Complexes"[mh] OR "Protein Aggregates"[mh] OR "Hormones, Hormone Substitutes, and Hormone Antagonists"[mh] OR "Enzymes and Coenzymes"[mh] OR "Carbohydrates"[mh] OR "Lipids"[mh] OR "Amino Acids, Peptides, and Proteins"[mh] OR "Nucleic Acids, Nucleotides, and Nucleosides"[mh] OR "Biological Factors"[mh] OR "Pharmaceutical Preparations"[mh] OR "Metabolism"[mh] OR "Cell Physiological Phenomena"[mh] OR "Genetic Phenomena"[mh] PubMed returns 11,447,521 abstracts. PMC returns, 1,720,266 documents, 509,722 of these are open access. We downloaded, parsed and concatenated 403,825 PMC open access documents into a single file `molecular_oa_pmc.tsv`. This is a 33GB TSV file with the following columns: File:Paragraph - a unique identifier for each paragraph SentenceId - the local number of the sentence in the document Sentence Text - tokenized text of the sentence (based on ClearTk's TokenAnnotator.java) Codes - <code>exLink</code> for the presence of a citation, <code>inLink</code> for the presence of link to a Figure Figures - Figure codes Headings - High level section of the paper Offset_Begin - offset of the start of the sentence within the paper Offset_End - offset of the start of the sentence within the paper We repeated the same process for PubMed abstracts to generate a 3.6G file (`molecular_oa_medline.tsv`) with three columns: Pubmed ID A Boolean value indicating whether the article is a review Text We concatenated the text columns of these two files into a single 30GB file (`molecular_oa.txt`) where each line is a single sentence and the text is fully tokenized. These three files are archived in `molecular_oa_raw_text.tar.gz`. <strong>Fasttext Embedding</strong> We trained a fasttext model on the raw training data (https://fasttext.cc/) using the standard `skipgram` parameter. A gzipped copy of the word embeddings is included in `fasttext.model.vec.gz`