Chemical Probes in Scientific Literature: A Pipeline for Target-Disease Evidence

This repository contains the main datasets generated for the manuscript, "Chemical Probes in Scientific Literature: Expanding and Validating Target-Disease Evidence." The datasets were derived using a Natural Language Processing (NLP) pipeline to systematically analyze open-access scientific literature for evidence of target-disease (T-D) associations mediated by chemical probes. This repository provides the primary results and aggregated datasets underpinning the manuscript's findings. This includes the dataset used for the pilot study, the main systematic dataset containing Probe-Target-Disease (P-T-D) triples annotated with Open Targets data, the novel target-disease pairs identified via the study's scoring scheme, and aggregated files for specific entities and filtering steps. These data can be used to reproduce the analyses in the manuscript, explore the landscape of chemical probe-based evidence, and identify novel T-D associations for further investigation. The complete workflow, including all intermediate files tracking the extraction process, can be reproduced by executing the main pipeline available on GitHub at: https://github.com/chembl/chemical_probes_lit File Descriptions This record contains the triplets found in the pilot study, the High Quality Chemical Probes dataset used in the systematic pipeline, and the main datasets extracted from the systematic pipeline: pilot_triplets.tsv: This file contains a preliminary or pilot subset of triplets, representing the data used for the initial validation and testing of the pipeline logic. probes_HQ.tsv: This file contains the curated dictionary of High-Quality (HQ) chemical probes. It serves as the reference input list for Step 1 of the extraction pipeline ner_all_triplet.tsv: This file contains the triplets identification from the raw data without checking for chemical to be chemical probes. ner_diseases.tsv: This file provides an aggregated view of all identified diseases. It maps unique diseaseIds to their human-readable diseaseName and includes counts of other entities (distinct PMIDs, targets, and probes) associated with each disease. ner_probes.tsv: This file provides an aggregated view of the chemical probes. It maps unique probeIds to their names and lists the unique counts of PMIDs, targets, and diseases found in association with each probe across the corpus. ner_targets.tsv: This file provides an aggregated view of the identified targets. It maps unique targetIds to their names and includes counts of the distinct PMIDs, probes, and diseases linked to each target. ner_target_disease_pairs.tsv: This dataset aggregates unique Target-Disease pairs found in the text. For each pair (defined by targetId and diseaseId), it lists the associated chemical probes (including their max clinical phase and the extracted sentences) and details the number of supporting articles. ner_target_disease_pairs_evd.tsv: The same data as in ner_target_disease_pairs.tsv but enriched with annotations such as other evidence in Open Target, date of evidence, therapeutic area, drug max phase, etc. Final file from all intermediate files in main pipeline. ner_probes_triplets_novel.tsv: This file contains the subset of novel triplets identified in the analysis. It details the target (targetId) and disease (diseaseId) associations, including the preferred Therapeutic Area (preferredTA), the count of supporting probes and PMIDs (probeId_count, pmid_count), and the maximum clinical phase (max_phase) identified. disease_term_blaklist.tsv: This file contains the excluded diseaseId for disease terms that where wrongly mapped due to misleading abbreviations. Such list is used as a preproccesing step for obtaining the novel cases (ner_probes_triplets_novel.tsv). The list was constructed after manual curation.