Prostate-TriMod: A Multi-Scale Tri-Modal Histology Dataset Integrating Morphology, Immune Patterns, and Clinical Outcomes

Accurate prognostic assessment of prostate cancer (PCa) requires an integrated understanding of tumor morphology and the immune microenvironment. We present Prostate-TriMod, a novel tri-modal histology dataset designed to integrate high-resolution morphology with spatial immune patterns and clinical outcomes. This dataset, generated from the Cell DIVE multiplexed imaging platform, consists of three synchronized modalities: (1) multiscale virtual H&E (vH&E) tiles (224px, 256px, 512px, and 2040px), (2) spatial tissue maps identifying cancerous/non-cancerous epithelial, stroma and immune cell populations (via TOPAZ and CAT models), and (3) text captions generated from single-cell data and patterns. The dataset includes comprehensive clinical annotations, including Grade Groups and biochemical recurrence (BCR) status. By providing high-fidelity alignment between morphology, spatial tissue maps, and textual descriptions, this resource enables the development of advanced multimodal AI models including vision-language models (VLMs) and deep learning frameworks for precision oncology. Version 2.0 extends Prostate-TriMod by adding processed single-cell protein-expression measurements from 717 tissue cores representing 212 patients. The added table contains 1,226,353 single-cell records and 12 proteins represented by 14 expression columns. P63 and CK5 include nuclear-mean and whole-cell-mean intensities, while NaK ATPase, S6, PCK26, AE1, AMACR, CD3, CD4, CD8, CD68, and FOXP3 include whole-cell-mean intensities. The measurements were generated from Cell DIVE multiplexed immunofluorescence images. Processed images were segmented to assign cell identifiers and spatial coordinates, and marker intensities were quantified by cellular compartment and for the whole cell. Cell-level quality control was applied before normalization to minimize batch effects, followed by log2 transformation to address skewed intensity distributions. The table also includes stable Prostate-TriMod identifiers, cell-centroid coordinates, phenotypic and immune-zone labels, biochemical-recurrence status, recurrence-free follow-up duration, and whole-tissue text captions. Because of staining-specificity limitations, CD4 measurements should be interpreted only in combination with CD3 measurements.