SA-Prot-annot

# SA-Prot-annot Dataset page: [https://huggingface.co/datasets/opendatalab/SA-Prot-annot](https://huggingface.co/datasets/opendatalab/SA-Prot-annot) SA-Prot-annot releases SeqStudio annotations for a UniProtKB-scale slice: about 1.2 million proteins spanning manually reviewed Swiss-Prot and computationally analyzed TrEMBL, in a single Parquet file at the repository root (`seqstudio_uniprot_1.2m.parquet`). ## SeqStudio annotation content SeqStudio is a generative protein functional annotation system. It is designed to approximate the integrative judgment of expert UniProt curators: orchestrating heterogeneous evidence, weighing reliability and specificity, reconciling cross-modal conflicts, and synthesizing mechanistic explanations—rather than treating annotation as a simple union of pattern-matching hits. Evidence conditioned on by SeqStudio includes, in line with the manuscript: sequence homology (BLAST against reviewed UniProt), domain and motif architecture (InterProScan, together with rule-based context such as UniRule where used in the pipeline), three-dimensional fold similarity (Foldseek), and membrane topology (TMHMM). Evidence items are semantically enriched (e.g. GO definitions, domain descriptions) before large language model–based generative reasoning, so outputs are grounded in retrieved signals rather than unconstrained parametric guessing. The pipeline produces a natural-language-style functional summary and structured predictions with per-field confidence (0–1) and explicit evidence provenance (`support`: motifs / GO terms / which tools contributed). Exact LLM and single- vs multi-turn configuration can differ between high-precision and high-throughput deployment modes; the on-disk schema is shared. ### What is stored in this dataset | Column | Role | |--------|------| | `seqStudioSummary` | Functional summary: integrated narrative of molecular mechanism, biological role, localization, and major structural features. | | `seqStudioComments` | Machine-readable JSON: `version`, `generatedAt`, and `predictions` over six functional dimensions (see below). Each dimension is typically an object with `value`, `confidence`, and `support` linking the claim to concrete evidence. | Parse with `json.loads` and read `obj["predictions"]`. Six prediction dimensions (manuscript / evaluation schema; JSON keys in current exports): | Dimension | Typical JSON key | Notes | |-----------|------------------|--------| | Protein family | `proteinFamily` | Family or superfamily assignment. | | Function | `function` | Molecular and biological role (text). *Some older records use `primaryFunction`.* | | Enzyme information | `enzymeInfo` | Enzyme flag, EC, catalytic description (often nested JSON). *Legacy alias: `catalyticActivity`.* | | Pathways | `pathways` | Pathway involvement (list or text). | | Subcellular location | `subcellularLocation` | Predicted localization (topology-informed when TMHMM is used). | | Structural class / architecture | `proteinStructure` | Domains, fold class, membrane protein flag, TM helix count, etc. *Legacy alias: `structuralClass`.* | Top-level fields `version` and `generatedAt` record the pipeline build and generation time for traceability. ### Relation to `toolResult` `toolResult` preserves raw outputs from the integrated bioinformatics tools (e.g. BLAST, InterProScan, Foldseek, TMHMM). SeqStudio consumes these as grounding; `seqStudioComments` holds evidence-conditioned structured predictions, auditable through `support` fields and side-by-side comparison with `toolResult`. ## Data file | File | Records (approx.) | Size (approx.) | Description | |------|-------------------|----------------|-------------| | `seqstudio_uniprot_1.2m.parquet` | 1,200,000 | 5.5 GB | UniProtKB mix: Swiss-Prot + TrEMBL; original UniProt fields, SeqStudio outputs, and `toolResult` | Composition (same split as the main SeqStudio dataset card): - Swiss-Prot: 573,661 (about 47.8%) — manually reviewed UniProtKB entries - TrEMBL: 626,339 (about 52.2%) — computationally analyzed entries Use column `data_source` to distinguish provenance labels such as `swiss`, `trembl5`, and `trembl4`. ## Quick start ```python import pandas as pd path = "hf://datasets/opendatalab/SA-Prot-annot/seqstudio_uniprot_1.2m.parquet" df = pd.read_parquet(path) print(len(df), df.columns.tolist()[:5]) ``` Using `datasets`: ```python from datasets import load_dataset ds = load_dataset( "opendatalab/SA-Prot-annot", data_files="seqstudio_uniprot_1.2m.parquet", ) print(ds["train"]) ``` ## Content summary - Coverage: about 1.2M UniProtKB proteins (Swiss-Prot + TrEMBL), with `data_source` marking origin. - Format: Parquet with 23 columns combining UniProt-style fields, SeqStudio prediction payloads, and bioinformatics tool results. - Highlights: see “SeqStudio annotation content” above for `seqStudioComments` / `seqStudioSummary`; `toolResult` aggregates supporting tool outputs. ## Column reference (23 columns) 1. `entryType` — entry type 2. `primaryAccession` — UniProt primary accession 3. `uniProtkbId` — UniProtKB ID 4. `entryAudit` — audit metadata (JSON string) 5. `annotationScore` — annotation score 6. `organism` — organism (JSON) 7. `proteinExistence` — protein existence evidence 8. `proteinDescription` — description (JSON) 9. `genes` — genes (JSON) 10. `comments` — comments (JSON) 11. `features` — features (JSON) 12. `keywords` — keywords (JSON) 13. `references` — references (JSON) 14. `uniProtKBCrossReferences` — cross-references (JSON) 15. `sequence` — sequence (JSON) 16. `extraAttributes` — extra attributes (JSON) 17. `seqStudioComments` — SeqStudio structured predictions (JSON: `predictions` with six dimensions—see “SeqStudio annotation content”; keys may be `function` / `enzymeInfo` / `proteinStructure` or legacy `primaryFunction` / `catalyticActivity` / `structuralClass`) 18. `seqStudioSummary` — integrated functional summary (text or JSON string, depending on export) 19. `toolResult` — tool outputs, e.g. InterProScan, BLAST (JSON) 20. `data_source` — provenance label (`swiss` / `trembl5` / `trembl4`, etc.) 21. `secondaryAccessions` — secondary accessions (JSON) 22. `organismHosts` — organism hosts (JSON) 23. `geneLocations` — gene locations (JSON) Example: `import json` then `json.loads(row["seqStudioComments"])` and read `["predictions"]`. ## Citation Please cite this dataset, UniProt, and the SeqStudio paper (Liu et al., *Generative reasoning emulating expert curation moves protein functional annotation beyond pattern matching at scale*) as appropriate once the reference is available. Example for the Hub release: ```bibtex @dataset{saprotannot2025, title={SA-Prot-annot: SeqStudio Annotations for UniProt 1.2M (Swiss-Prot + TrEMBL)}, author={OpenDataLab}, year={2025}, url={https://huggingface.co/datasets/opendatalab/SA-Prot-annot} } ``` ## License This dataset is released under the [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/) license: you may share and adapt the material, provided you give appropriate credit, indicate if changes were made, and do not add legal terms that restrict others from doing anything the license permits. The underlying protein records and many raw fields originate from [UniProt](https://www.uniprot.org/); use of this dataset should remain consistent with UniProt’s own terms and citation expectations in addition to CC BY 4.0.