Protein Set Transformer: A protein-based genome language model to power high diversity viromics

Exponential increases in microbial and viral genomic data demand transformational advances in scalable, generalizable frameworks for their interpretation. Standard homology-based functional analyses are hindered by the rapid divergence of microbial and especially viral genomes and proteins that significantly decreases the volume of usable data. Here, we present Protein Set Transformer (PST), a protein-based genome language model that models genomes as sets of proteins without considering sparsely available functional labels. Trained on >100k viruses, PST outperformed other homology- and language model-based approaches for relating viral genomes based on shared protein content. Further, PST demonstrated protein structural and functional awareness by clustering capsid-fold-containing proteins with known capsid proteins and uniquely clustering late gene proteins within related viruses. Our data establish PST as a valuable method for diverse viral genomics, ecology, and evolutionary appl..., Genomes used to create this dataset are publicly available, and all data within this dataset were generated by the study. See the manuscript for details., , # Data from: Protein Set Transformer: A protein-based genome language model to power high diversity viromics

These datasets are associated with the manuscript \"Protein Set Transformer: A protein-based genome language model powers high diversity viromics\". ([https://doi.org/10.1101/2024.07.26.605391](https://doi.org/10.1101/2024.07.26.605391))

## Dataset descriptions

This statement is relevant for the file references in the manuscript code: We refer to the processed data directly used for figure making as \"**Source Data**\". Major datasets that are used throughout the manuscript are referred to as \"**Datasets**\". Due to the size of the \"datasets\", we have provided each dataset folder as individual tarballs. \"Supplementary Data\" and \"Supplementary Tables\" are manuscript-associated tables, with \"Supplementary Data\" being too large otherwise be manuscript tables.

### Main data

| File | Description ..., , **Changes after Sep 19, 2024:**

* Added
* `foldseek_databases.tar.gz`
* Precomputed foldseek 3Di databases for each test dataset
* `PST-TL-P__small.ckpt.gz`
* New pretrained model checkpoint for model trained with triplet loss and tuned with protein diversity groups
* `PST-TL-P__large.ckpt.gz`
* New pretrained model checkpoint for model trained with triplet loss and tuned with protein diversity groups
* `PST-MLM.tar.gz`
* New pretrained model checkpoints for models trained with masked language modeling loss
* `PST_training_set_PST-TL-P__large_protein_embeddings.h5`
* `IMGVRv4_test_set_PST-TL-P__large_protein_embeddings.h5`
* `MGnify_set_PST-TL-P__large_protein_embeddings.h5`
* `PST_training_set_PST-TL-P__small_protein_embeddings.h5`
* `IMGVRv4_test_set_PST-TL-P__small_protein_embeddings.h5`
* `MGnify_set_PST-TL-P__small_protein_embeddings.h5`

* Changed
* `esm-large_protein_embeddings.tar.gz`
* Now part of `esm_embeddings.tar.gz`
* Includes...