Who Actually Breaks on ALL-CAPS? Casing Sensitivity in Embedding-Based Biomedical Ontology Alignment — Benchmark code and derived artefacts (13 open-weight models on NCBI Taxonomy, microbiome and MIMIC-IV)
收藏资源简介:
Benchmark code, prebuilt FAISS indices, and openly-redistributable derived results accompanying the article “Who Actually Breaks on ALL-CAPS? Casing Sensitivity in Embedding-Based Biomedical Ontology Alignment” (submitted to Health Information Science and Systems). Thirteen open-weight sentence-transformer models plus three baselines (BM25, RapidFuzz, ETE3) are evaluated for zero-shot retrieval of organism names against the full NCBI Taxonomy (2.69M canonical names), with input casing isolated as a controlled variable via a matched-query design (n=377). Using the matched subset, 12 of 16 systems are casing-invariant (delta Hits@1 = 0.000, including the case-insensitive ETE3 control); casing is the dominant failure mode only for case-sensitive systems (RapidFuzz, ClinicalBERT, Bio_ClinicalBERT). SapBERT fails structurally due to species-epithet collapse. MIMIC-IV-derived query sets and per-query result files are NOT included here: they are governed by the PhysioNet Data Use Agreement and are available to credentialed PhysioNet users, who can regenerate them with the provided scripts. See DATA_AVAILABILITY.md.



