Addressing reproducibility in single-laboratory phenotyping experiments [Supplementary Materials]

There has been a growing concern that preclinical research is failing to replicate experimental results. The problem is especially relevant to phenotyping genetically engineered mouse lines, a central strategy for discovering mammalian gene function and developing animal models of disease. We introduce a statistical method to estimate whether a single-laboratory discovery is likely to replicate in other laboratories, based on previously-estimated variability of genotype × laboratory interaction in large phenotyping databases. Future single-lab experiments can be used to further update the estimation, making the proposed method a true community effort. We validate the method by combining several datasets into the most inclusive analysis conducted to date of across-laboratory replication in mouse phenotyping, and estimate that with current single laboratory analysis 19%–41% of non-replicable phenotypic differences between genotypes are still published as “discoveries”. Applying our proposed method would reduce this rate to 3.3%–9%, close to the intended 0.05, both for testing and confidence intervals. This fileset includes the analysis results, the R and Sweave scripts to reproduce them, the input datasets, output tables with the estimated variances, the phenotypic measures transformations and the paper figure's source data.

临床前研究难以重复实验结果的问题日益受到关注。该问题在基因工程小鼠品系的表型分型（phenotyping）领域尤为突出——而表型分型正是解析哺乳动物基因功能、构建疾病动物模型的核心策略。本研究基于大型表型数据库中已估算得到的基因型-实验室交互作用变异度，提出一种统计方法，用于评估单个实验室的研究发现能否在其他实验室中重复。未来的单实验室实验可用于进一步更新该估算结果，使本方法真正成为一项社区协作的研究成果。本研究通过整合多个数据集，开展迄今为止规模最大的小鼠表型跨实验室重复性分析，以此验证所提方法；经估算，当前采用单实验室分析时，有19%~41%的基因型间表型差异为不可重复的结果，却仍被作为「发现」发表。若应用本研究提出的方法，该比例可降至3.3%~9%，接近检验与置信区间预设的0.05显著性水平。本数据集包含分析结果、用于复现分析的R与Sweave脚本、输入数据集、含估算方差的输出表格、表型测量值转换方法以及论文配图的源数据。