DALPHIN: A multicentric open benchmark for digital pathology AI copilots

The digital pathology AI copilot benchmark (DALPHIN) dataset is a multicentric, open benchmark for evaluating AI copilots in digital pathology. DALPHIN consists of 300 cases collected across six healthcare institutions in six countries, covering 130 diagnoses from 14 pathology subspecialties, including non-neoplastic entities and rare cancers.

The benchmark includes 1,236 histopathology images (low-resolution whole-slide images and higher-resolution regions of interest) and 1,757 questions across six tasks: tissue/organ recognition, neoplastic status, neoplastic behavior (benign, malignant, in situ, or uncertain), diagnosis, and case-specific multiple-choice and free-response questions.

The images and questions are publicly available via this Zenodo record. Example code to run models on the benchmark and generate responses is provided in the associated GitHub repository. The reference answers are not publicly released but are sequestered and indirectly accessible on the Grand Challenge platform, where submissions are evaluated and ranked on public leaderboards to ensure fair and reproducible evaluation of pathology AI copilots.