Data from: Detecting the anomaly zone in species trees and evidence for a misleading signal in higher-level skink phylogeny (Squamata: Scincidae)

The anomaly zone, defined by the presence of gene tree topologies that are more probable than the true species tree, presents a major challenge to the accurate resolution of many parts of the Tree of Life. This discrepancy can result from consecutive rapid speciation events in the species tree. Similar to the problem of long-branch attraction, including more data via loci concatenation will only reinforce the support for the incorrect species tree. Empirical phylogenetic studies often employ coalescent-based species tree methods to avoid the anomaly zone, but to this point these studies have not had a method for providing any direct evidence that the species tree is actually in the anomaly zone. In this study, we use 16 species of lizards in the family Scincidae to investigate whether nodes that are difficult to resolve place the species tree within the anomaly zone. We analyze new phylogenomic data (429 loci), using both concatenation and coalescent-based species tree estimation, to locate conflicting topological signal. We then use the unifying principle of the anomaly zone, together with estimates of ancestral population sizes and species persistence times, to determine whether the observed phylogenetic conflict is a result of the anomaly zone. We identify at least three regions of the Scindidae phylogeny that provide demographic signatures consistent with the anomaly zone, and this new information helps reconcile the phylogenetic conflict in previously published studies on these lizards. The anomaly zone presents a real problem in phylogenetics, and our new framework for identifying anomalous relationships will help empiricists leverage their resources appropriately for investigating and overcoming this challenge.

异常区（anomaly zone）被定义为存在比真实物种树更具概率性的基因树拓扑结构的区域，这对生命之树诸多分支的精准解析构成了重大挑战。此类拓扑差异可能源于物种树中发生的连续快速物种形成事件。与长枝吸引（long-branch attraction）问题类似，通过基因座串联获取更多数据，只会强化对错误物种树的支持度。实证系统发育研究通常采用基于溯祖的物种树推断方法以规避异常区，但截至目前，此类研究尚无方法能够直接证明物种树确实处于异常区中。本研究以石龙子科（Scincidae）的16个蜥蜴物种为研究对象，探究难以解析的节点是否将物种树置于异常区范围内。我们通过串联法与基于溯祖的物种树估计两种策略，分析新获得的系统基因组数据（429个基因座），以定位冲突的拓扑信号。随后，我们结合异常区的统一原理，以及祖先种群大小与物种存续时间的估计值，判断观测到的系统发育冲突是否由异常区导致。我们至少在石龙子科系统发育的三个区域中发现了与异常区相符的种群遗传学特征，这一新发现有助于调和此前针对这些蜥蜴的已发表研究中存在的系统发育冲突。异常区确实是系统发育学中的一项真实难题，我们提出的识别异常关系的新框架，将帮助实证研究者合理调配资源，以开展相关研究并攻克这一挑战。