Hidden-uncertainty Assessment via Non-verbalized Signatures for Medical QA - Dataset

Associated pre-print paper version: https://www.techrxiv.org/doi/full/10.36227/techrxiv.176800997.72656064/v1This dataset contains raw inference outputs from five large language models (LLMs) on ten medical question-answering benchmarks (N=21,324 samples). Each sample includes: - Full chain-of-thought reasoning traces - Model predictions and ground-truth labels - Verbalized confidence scores (0-100) - Token-level log probabilities with top-20 alternatives for each token Models: GPT-oss-120B (medium/high reasoning), DeepSeek-R1-Distill-32B, Qwen3-32B, Olmo-3-32B-Think Datasets: MedQA, MedMCQA, PubMedQA, MMLU, MMLU-Pro, MedBullets, MedExQA, AfriMedQA, MedXpertQA-R, MedXpertQA-U The dataset enables research on uncertainty quantification, error detection, and reasoning analysis in medical AI systems. It accompanies the paper "Hidden-uncertainty Assessment via Non-verbalized Signatures for Medical QA" (D'Urso, 2026). Additionally includes a robustness subset with 5 independent inference runs on 300 samples per dataset for self-consistency analysis.