GEOAgent Knowledge Base: An Integrated and Self-Contained Resource of SQLite Metadata, Vector Embeddings, and Local Re-ranking Models for Automated GEO Data Reuse

Overview This repository contains the core knowledge base for GEOAgent, an intelligent assistant designed to automate the discovery and analysis of biomedical datasets. The database integrates high-fidelity metadata from 180,000+ GEO series, 5 million+ samples, and 84,000+ PubMed abstracts (updated as of April 2026) into a multi-modal retrieval system. Core Components SQLite DB (Structured Metadata): A relational database optimized for high-speed filtering. It features a multi-table schema (including gse_metadata, gsm_metadata, and pubmed_metadata) with B-tree indexing for range/equality queries and FTS5 indexing for full-text search of study titles and summaries. Vector DB (Semantic Embeddings): An unstructured knowledge store generated via Hierarchical Semantic Chunking (HSC). Using the Nomic Embed v2 model (768 dimensions), raw metadata is systematically partitioned into five distinct semantic zones to enable granular similarity matching, where gse_core captures study design and research rationale, gse_sample encompasses sample-specific metadata and clinical/biological attributes, gse_protocol extracts experimental protocols including sample treatment, growth, and extraction, gse_processing documents downstream data processing and computational workflows, and pub_core incorporates rich bibliographic context via publication titles and abstracts. BGE-Reranker (Deep Ranking): An industry-standard Cross-Encoder model (BAAI/bge-reranker-v2-m3) co-packaged to enable full offline execution. Deployed in the final stage of the hybrid retrieval pipeline, it performs deep semantic re-ranking of dataset candidates to guarantee the highest precision for complex natural language queries. Technical Workflow Integration The database is specifically engineered to support the GEOAgent 5-stage pipeline: Intent Parsing: LLM-based extraction of research goals. Hybrid Retrieval: Concurrent SQL Hard Filtering (for structured attributes) and Semantic Matching (for unstructured context). Logical Filtering & Reranking: Final validation and precision sorting of results. Application & Compatibility This database serves as the foundational data asset for automated technology modality identification and cross-platform sample pairing (e.g., ChIP-seq IP/Input control matching and single-cell multi-omics linkage). ,This database serves as the foundational data asset for automated technology modality identification and cross-platform sample pairing (e.g., ChIP-seq IP/Input control matching and single-cell multi-omics linkage). It offers native, turnkey integration with bioStream—an industrialized, containerized Nextflow workflow platform—enabling automated and highly reproducible standardized processing for 6 major omics types: RNA-seq, scRNA-seq, ATAC-seq, scATAC-seq, ChIP-seq, and scMultiome. Links & Repositories Core Agent Software: GitHub - JiekaiLab/GEOAgent Multi-Omics Processing Pipeline: GitHub - JiekaiLab/bioStream Links & Ecosystem Web Application Portal: http://geoagent.ccla.ac.cn/— Try GEOAgent directly in your browser with our turnkey web interface. GEOAgent Core & Deployment Source: GitHub - JiekaiLab/GEOAgent — Repository for the desktop client and intelligent agent backend. Multi-Omics Processing Pipeline: GitHub - JiekaiLab/bioStream — Nextflow workflow backend for automated primary analysis and sequencing data quantification.# 概览 本仓库为GEOAgent的核心知识库，GEOAgent是一款旨在自动化发现与分析生物医学数据集的AI智能体（AI Agent）。该数据库整合了截至2026年4月更新的18万+GEO数据集系列、500万+样本以及8万+PubMed摘要的高保真元数据，构建为多模态检索系统。 ## 核心组件 ### SQLite数据库（结构化元数据） 一款针对高速筛选优化的关系型数据库。其采用多表架构（包含`gse_metadata`、`gsm_metadata`与`pubmed_metadata`三张表），配套B树索引以支持范围/等值查询，并通过FTS5索引实现研究标题与摘要的全文检索。 ### 向量数据库（语义嵌入） 通过层级语义分块（Hierarchical Semantic Chunking, HSC）生成的非结构化知识存储库。依托Nomic Embed v2模型（768维），原始元数据被系统性划分为五大独立语义域，以实现细粒度相似性匹配：其中`gse_core`涵盖研究设计与研究基本依据，`gse_sample`包含样本专属元数据及临床/生物学属性，`gse_protocol`提取包括样本处理、培养与提取在内的实验方案，`gse_processing`记录下游数据处理与计算工作流，`pub_core`则通过出版物标题与摘要整合丰富的文献背景信息。 ### BGE重排序器（深度排序） 一款行业通用的交叉编码器（Cross-Encoder）模型（BAAI/bge-reranker-v2-m3），已预装以支持完全离线运行。该模块部署于混合检索管线的最终阶段，可对数据集候选结果执行深度语义重排序，确保复杂自然语言查询下的最高检索精度。 ## 技术工作流集成 本数据库专为支持GEOAgent的五阶段工作流设计： 1. **意图解析**：基于大语言模型（Large Language Model, LLM）的研究目标提取。 2. **混合检索**：同时执行SQL结构化硬筛选（针对结构化属性）与语义匹配（针对非结构化上下文）。 3. **逻辑筛选与重排序**：对检索结果进行最终验证与精度排序。 ## 应用与兼容性 本数据库作为核心数据资产，可支持自动化技术模态识别与跨平台样本配对任务（例如ChIP-seq的IP/Input对照匹配，以及单细胞多组学联动分析）。 本数据库作为核心数据资产，可支持自动化技术模态识别与跨平台样本配对任务（例如ChIP-seq的IP/Input对照匹配，以及单细胞多组学联动分析）。 其原生支持与bioStream——一款工业化容器化的Nextflow工作流平台——的开箱即用集成，可实现六大组学类型的自动化、高可复现标准化处理：RNA-seq、scRNA-seq、ATAC-seq、scATAC-seq、ChIP-seq及scMultiome。 ## 链接与代码仓库 - **核心智能体软件**：GitHub - JiekaiLab/GEOAgent - **多组学处理工作流**：GitHub - JiekaiLab/bioStream ## 链接与生态系统 - **Web应用门户**：http://geoagent.ccla.ac.cn/ —— 可通过我们的一键式网页界面直接在浏览器中试用GEOAgent。 - **GEOAgent核心与部署源码**：GitHub - JiekaiLab/GEOAgent —— 桌面客户端与智能助手后端的代码仓库。 - **多组学处理工作流**：GitHub - JiekaiLab/bioStream —— 用于自动化原始数据分析与测序数据定量的Nextflow工作流后端。