CAMT paper: analysis scripts, results and audit coding sheet
收藏资源简介:
This record contains the analysis scripts, result files and audit coding sheet that accompany the manuscript "A clustered AUC maximum test (CAMT) for 'best of K' model comparisons on patient-clustered diagnostic data" (Gul, A.). The manuscript studies the claim that the best of K candidate models has a higher AUC than a baseline on a test set with several images or records per patient. It proposes CAMT, a patient-level test that corrects for within-patient clustering and for selection of the winner, and evaluates it by simulation, on five public datasets (BreakHis, Parkinson's voice recordings, PAD-UFES-20, PTB-XL and the Kermany retinal OCT archive) and in a pilot audit of published claims. Contents: scripts/: simulation studies, experiments on the five datasets, the audit scripts and the research version of the test. data/: result files and run logs (exact nulls, power, simulations, within-patient correlation, second-metric and replication studies, numerical checks of the theory). data/audit/: the audit of published claims, with the coded entries and the sentence supporting each value, the audit results and the screening records. data/deep/: test-set scores and training logs of the deep models on PAD-UFES-20. paper/honest_inference/: scripts that build the tables and figures, and the numbers behind each figure. The original datasets, extracted features and the full texts of the screened papers are not included; the README lists the sources and the scripts that download and prepare the data. The released implementation of the test is a separate record (doi:10.5281/zenodo.23771271; Python package camt-test on PyPI). Code is released under the MIT licence; result files and the coding sheet under CC BY 4.0.



