Benchmark Dataset and Experimental Results for MS/MS Spectrum Retrieval with SpecRoFormer

This repository provides a curated collection of experimental results and benchmark data for single-spectrum tandem mass spectrometry (MS/MS) retrieval tasks. It is designed to facilitate transparent evaluation and reproducibility of machine learning models under cross-dataset and cross-instrument settings. The data are collected from multiple widely used public mass spectrometry repositories, including GNPS, MoNA, MassBank, MassSpecGym, and MTBLS1572. Spectra are organized according to instrument types (e.g., Orbitrap and QTOF) to support systematic generalization analysis. To ensure a rigorous and unbiased evaluation protocol, a strict non-overlapping training–testing setup is adopted. Models are trained exclusively on the GNPS-Orbitrap subset, while all remaining datasets and instrument types are used solely for evaluation. This setup enables a realistic assessment of model generalization across different experimental conditions and data sources. The underlying benchmark dataset used in this study is publicly available at:👉 https://doi.org/10.5281/zenodo.19368733 This dataset provides the standardized data splits and preprocessing necessary to reproduce all experiments reported in this repository. The repository includes several categories of experimental outputs: ROC & AUC Analysis: Evaluation of global ranking quality across datasets and instruments Embedding–Structure Consistency: Analysis of the alignment between learned embedding similarity and chemical structural similarity Clustering Analysis: Visualization and quantitative evaluation of embedding space structure Attention Analysis: Investigation of model attention patterns for interpretability Dataset Statistics: Summary statistics and distributions of the underlying datasets Results & Ablation: Main experimental results and ablation studies of SpecRoFormer, including performance statistics and released model weights All results are generated using the SpecRoFormer model, a Transformer-based framework designed for MS/MS spectrum representation learning. The model incorporates relative m/z relationship modeling, explicit precursor conditioning, and fragmentation-aware augmentation to improve robustness and generalization. The data are organized into multiple folders corresponding to different evaluation aspects (e.g., ROC&AUC, clustering, attention analysis), allowing reviewers to easily navigate and inspect specific results. This repository is intended to support peer review, reproducibility, and further research in mass spectrometry-based representation learning and retrieval.