PubMiner AI – Automatic Knowledge Extraction from Scientific Publications

**PubMiner AI**

**Automatic Knowledge Extraction from Scientific Publications**

**Description**
Academic research described in scientific publications is essential for advancing drug discovery, as it uncovers new biological targets, mechanisms, and therapeutic strategies. To unlock and utilize this information, different AI methods can be used to extract key insights from vast volumes of literature. These approaches enable researchers to efficiently identify promising drug targets and innovations, facilitating collaboration and accelerating drug development. However biomedical researchers cannot implement such advanced data processing and information extraction pipelines. They need to rely on skilled data scientists and engineers who (with the necessary pre-built smart assistant and reference data sets) can quickly build and automate flexible workflows capable of extracting complex relationships between various objects of scientific interest.

**PubMiner AI** is a scalable, AI-enabled workflow automating the extraction of valuable knowledge from a large body of scientific publications. It is an implementation of Retrieval Augmented Generation (RAG) using structured graph knowledge in the retrieval step. By integrating data from [PubMed](https://pubmed.ncbi.nlm.nih.gov), [SemMedDB](https://lhncbc.nlm.nih.gov/ii/tools/SemRep_SemMedDB_SKR/dbinfo.html), and Ontotext’s [LinkedLifeData Inventory](https://www.ontotext.com/solutions/healthcare-and-life-sciences/linked-life-data-inventory/?ref=menu) powered by [GraphDB](https://www.ontotext.com/products/graphdb/), PubMinder AI distills the potential answer space into a view of the merged underlying biomedical datasets. This is then fed to a large language model (LLM**) to produce a focused knowledge graph formalizing potentially novel knowledge in the specific domain. Built on the principles of [FAIR](https://www.go-fair.org/fair-principles/), such knowledge representation is directly embeddable into downstream information systems. Researchers can leverage this approach to transform vast amounts of medical data into actionable knowledge through AI-powered insights, graphical representations, and customizable queries.

Using the PubMiner AI as a blueprint, any skilled data scientist with basic knowledge of semantic technologies (knowledge graphs, RDF, SPARQL, ontologies, and semantic models) can build a targeted workflow that can extract complex relations between biomedical entities from scientific literature. The workflow allows 1) retrieval of a very targeted subset of documents containing relevant information for the particular use case; 2) identification of biomedical concepts and the relations between them; and 3) building of a subgraph representing the extracted knowledge normalized to reference data sets.

** In this workflow Meta-Llama-3.1-70B-Instruct served by Databricks is used.

**Key Features**
* Integrated Data Sources: Combines data from PubMed, SemMedDB, and Ontotext’s LinkedLifeData Inventory powered by GraphDB into a highly scalable biomedical knowledge base.
* AI-Powered Knowledge Creation: Uses large language models to analyze cross-referenced biomedical data and generate in-depth insights.
* Knowledge Graph Generation: Automatically creates knowledge graphs to visualize and explore relationships between the biomedical entities in scope: diseases and genes.
* GraphDB Integration: Leverages GraphDB to enhance data structuring and querying, facilitating more complex insights from interconnected data.
* Customizable Data Subsets: Allows users to filter and focus on specific areas of interest for personalized knowledge discovery.
* Automated Report Generation: Summarizes research findings and trends into comprehensive, digestible reports.
* Enhanced Query Capabilities: Natural language queries enable intuitive, AI-driven exploration of large biomedical datasets.
* Biomarker Discovery: Facilitates the mining of scientific literature to identify potentially novel biomarkers for diseases or conditions.

**Use Cases**
* Medical Research: Conduct thorough literature reviews, discover new relationships, and visualize findings with knowledge graphs.
* Pharmaceutical Development: Identify emerging biomarkers, drug interactions, and treatment trends to accelerate drug discovery and clinical research.
* Healthcare Analytics: Use AI-powered insights and knowledge graphs to uncover new relationships between diseases, treatments, and outcomes.

**Target Audience**

Biomedical researchers, pharmaceutical companies, healthcare professionals, and data scientists looking to integrate AI with knowledge graphs for enhanced biomedical literature analysis and knowledge discovery.

**Packaging / Contents**
PubMiner AI contains:
* Main notebook demonstrating sample workflow for knowledge extraction.
* Complementary notebook which serves as a detailed guideline to explore LinkedLifeData Inventory (LLDI) for retrieving linked datasets and concepts of interest.
* Sample dataset of PubMed articles to execute the workflow against.

**How to run this workflow**
* Request a 30-days free trial access to Ontotext’s LinkedLifeData Inventory and Biomedical Entity Linker from the [Ontotext website](https://www.ontotext.com/pubminer-ai/). Enter "Databricks accelerator PubMiner AI" as context for your inquiry.
* Import this listing.
* As Meta-Llama-3.1-70B-Instruct pay-per-token, served by Databricks, is used in this workflow, it’s necessary to have an active Databricks token. See more [here](https://docs.databricks.com/en/dev-tools/auth/pat.html).
* Use this notebook to run all steps and explore the results of the workflow.

**Get access**

This workflow relies on Ontotext's LinkedLifeData Inventory and Biomedical Entity Linking Service, which require logging in. To obtain the credentials for these services, please register at the [Ontotext's website](https://www.ontotext.com/pubminer-ai/) and enter "Databricks accelerator PubMiner AI" as context for your inquiry.