Phylogenies and diversification rates: variance cannot be ignored

A recent pair of articles published in the journal Evolution presented a test for assessing the validity of hierarchical macroevolutionary models. The premise of the test is to compare numerical point estimates of parameters from two levels of analysis; if the estimates differ, the hierarchical model is purportedly flawed. The articles in question (Meyer and Wiens 2017; Meyer et al. 2018) apply their proposed test to BAMM, a scientific software program that uses a Bayesian mixture model to estimate rates of evolution from phylogenetic trees. The authors use BAMM to estimate rates from large phylogenies (n > 60 tips) and they apply the method separately to subclades within those phylogenies (median size: n = 3 tips); they find that point estimates of rates differ between these levels and conclude that the method is flawed, but they do not test whether the observed differences are statistically meaningful. There is no consideration of sampling variation and its impact at any level of t...