Addressing reproducibility in single-laboratory phenotyping experiments [Supplementary Materials]

There has been a growing concern that preclinical research is failing to replicate experimental results. The problem is especially relevant to phenotyping genetically engineered mouse lines, a central strategy for discovering mammalian gene function and developing animal models of disease. We introduce a statistical method to estimate whether a single-laboratory discovery is likely to replicate in other laboratories, based on previously-estimated variability of genotype × laboratory interaction in large phenotyping databases. Future single-lab experiments can be used to further update the estimation, making the proposed method a true community effort. We validate the method by combining several datasets into the most inclusive analysis conducted to date of across-laboratory replication in mouse phenotyping, and estimate that with current single laboratory analysis 19%–41% of non-replicable phenotypic differences between genotypes are still published as “discoveries”. Applying our proposed method would reduce this rate to 3.3%–9%, close to the intended 0.05, both for testing and confidence intervals. This fileset includes the analysis results, the R and Sweave scripts to reproduce them, the input datasets, output tables with the estimated variances, the phenotypic measures transformations and the paper figure's source data.