Data and code from: Correcting for complexity: Incorporating trait-numbers enhances the performance of EMMLi in investigating modularity

Adams and Collyer (2019) evaluated the statistical performance of several
approaches for quantifying morphological modularity and found that EMMLi
had inflated type I error rates and a bias towards more complex models
compared to the Covariance Ratio (CR) approach. They suggested that this
may have been at least partly driven by the fact that AICc values from
EMMLi do not incorporate trait numbers, but this was not verified. Here I
present a performance analysis of a trait-number corrected EMMLi approach
(“EMMLip”), showing that this ameliorates rates of false discovery and
produces conservative results that favor less complex models. The
corrected EMMLi approach was effective at differentiating models of
modularity with varying between- and within-module covariation especially
when effect size or dataset size were sufficiently large. While CR tests
remained more effective at specifically detecting overall modularity, I
found that CR tests are sensitive to varying within/between module
covariation, and in some cases had inflated model misspecification between
2- and 3-module hypotheses. With this minor correction (albeit
incomplete), the combination of EMMLip and CR tests becomes the best
available toolkit for detecting and contrasting modularity hypotheses.
This toolkit is however still imperfect, and I discuss future avenues for
improvements.

Adams与Collyer（2019）评估了多种量化形态模块化（morphological modularity）方法的统计性能，发现相较于协方差比（Covariance Ratio, CR）方法，EMMLi存在膨胀的一类错误率（type I error rates），且倾向于选择更复杂的模型。他们推测该现象至少部分源于EMMLi输出的修正赤池信息准则（AICc）未纳入性状数量，但该推测尚未得到验证。本研究针对经过性状数校正的EMMLi方法（下称"EMMLip"）开展性能分析，结果表明该校正方法可降低错误发现率，并生成偏向更简单模型的保守性结果。经校正的EMMLi方法能够有效区分具有不同模块间与模块内协方差结构的模块化模型，尤其当效应量（effect size）或数据集规模（dataset size）足够大时。尽管CR检验在特异性检测整体模块化方面仍表现更优，但本研究发现CR检验对模块内/模块间协方差的变化较为敏感，在部分场景下，其在2模块与3模块假设间会出现膨胀的模型设定误差（model misspecification）。通过这一（虽不完整的）小幅校正，EMMLip与CR检验的组合成为当前检测与对比模块化假设的最优可用工具集。不过该工具集仍存在缺陷，本文最后讨论了未来的改进方向。