Scoutknife: A naïve, whole genome informed phylogenetic robusticity metric

The phylogenetic bootstrap, first proposed by Felsenstein in 1985, is a critically important statistical method in assessing the robusticity of phylogenetic datasets. Core to its concept was the use of pseudosampling - assessing the data by generating new replicates derived from the initial dataset that was used to generate the phylogeny. In this way, phylogenetic support metrics could overcome the lack of perfect, infinite data. With infinite data, however, it is possible to sample smaller replicates directly from the data to obtain both the phylogeny and its statistical robusticity in the same analysis. Due to the growth of whole genome sequencing, the depth and breadth of our datasets have greatly expanded and are set to only expand further. With genome-scale datasets comprising thousands of genes, we can now obtain a proxy for infinite data. Accordingly, we can potentially abandon the notion of pseudosampling and instead randomly sample small subsets of genes from the thousands of genes in our analyses. Here, we introduce Scoutknife, a jackknife-style subsampling implementation that generates 100 datasets by randomly sampling a small number of genes from an initial large-gene dataset to jointly establish both a phylogenetic hypothesis and assess its robusticity. Using 18 previously published datasets and 100 simulation studies, we show that Scoutknife is conservative and informative as to conflicts and incongruence across the whole genome, without the need for subsampling based on traditional model selection criteria. Methods Dataset Construction and Analysis For both real and simulation analyses (for details see below), 100 genes were randomly selected 100 times from the source datasets, generating 100 100-gene concatenated sample datasets using the Scoutknife Package (https://github.com/JFFleming/Scoutknife). The Scoutknife script package requires catsequences23 to be installed as a prerequisite, available at (https://github.com/ChrisCreevey/catsequences). Phylogenies for each Scoutknife dataset were constructed under IQ-Tree v1.6.1224 using Modeltest25, with a separate model applied to each gene and no partition merging. As a data density-based technique, Scoutknife might be expected to perform better in high data density scenarios where partitions can be comfortably merged. As such, this was intended to limit the efficacy of Scoutknife further and test its performance under a scenario with more highly variable best fit models than might be expected under normal conditions, whilst conserving computational effort considering the large number of test datasets and simulations. To further facilitate parallelization of the analyses, the phylogenetic analyses of the datasets were submitted using Scoutknifette (https://github.com/Togtja/scoutknifette). Scoutknifette is a custom HPC webhook for the group messaging service Discord that can be easily modified for any HPC tasks that require multiple submission batches and queue tracking. The trees produced by each Scoutknife sample dataset were then concatenated into a single treelist file (see Supplemental Information), and a consensus tree was constructed using bpcomp, available in Phylobayes26, by using a burnin of 0 and a sampling rate of 1, sampling each tree in the treelist. Trees were constructed as both 70% strict consensus and 50% majority consensus trees, and the results were compared. In two cases (Araneae and Lepidoptera), 30% plurality consensus trees were constructed using the same method, to further explore the data, as explained in the results and discussion section. In a single case (Actinopterygii), the low occupancy of two species in particular (Muraenesox cinerus with 1 gene and Scomber scombrus with 15 genes across the entire dataset of 1105) meant that many of the Scoutknife samples did not contain representatives from these taxa. To address this, we used sumtrees.py v 4.5.2, part of the DendroPy package27, as it is capable of building consensus trees from tree lists containing a variable number of taxa. The Quartet Similarity, Quartet Divergence, Node Conflict, Node Agreement, Strict Joint Assertions, Semi-Strict Joint Assertions, Symmetric Difference, Marczewski-Steinhaus, Steel-Penny and Overall Similarity were measured with reference to the previously published topology of the 250 most informative genes of that dataset, as selected by GeneSortR20. In the case of the simulated datasets, the topology of the 250 most informative genes of the original source dataset, Milla et al (2020), as selected by GeneSortR20, 28, was used. These similarity metrics were calculated using the ‘Quartet’ Library available in R29. Real Test Datasets To assess the efficacy of Scoutknife, we examined 18 real data datasets28, 30-46, those used in a similar benchmarking study by GeneSortR20. These datasets range from 1049 to 5105 genes and from 30 to 332 taxa in size, comprising studies of animals, plants and fungi (Table 1). In contrast to the prior study, genes with less than 50% occupancy were not removed: Scoutknife should show decreased performance at low occupancy levels, as it relies on data density, and so this should give a clearer picture of how the methodology performs across a variety of real datasets. The resultant tree topologies were then compared to the topology recovered by analysing the most informative 250 genes, as determined by GeneSortR20, to assess whether the same topological hypothesis was resolved by the Scoutknife Consensus Tree. Simulation Datasets To further assess the efficacy of Scoutknife, we generated 100 simulation datasets using the Alignment Mimic function of AliSim, as implemented in IQTree v2.2.047, 48. For this, 100 simulations were independently created for each gene in the Milla et al (2020)28 Heliozelidae dataset, as it represented a small-sized dataset of those within our real data study, at 1049 genes, and as such should have presented a challenge for Scoutknife. Furthermore, AliSim’s alignment mimic47 allows us to generate alignment datasets that mimic real genes, complete with low occupancy and reasonable variations in alignment length. Alisim was implemented with the following command: iqtree2 –alisim < Output > -s <Gene> --num-alignments 100 For each set of 1049 simulated genes, 100 100-gene Scoutknife datasets were constructed, and then analysed using IQ Tree as with the real datasets. The Quartet Similarity, Quartet Divergence, Node Conflict, Node Agreement, Strict Joint Assertions, Semi-Strict Joint Assertions, Symmetric Difference, Marczewski-Steinhaus, Steel-Penny and Overall Similarity were then measured with reference to the previously published topology generated by analysing the 250 most informative genes of the Milla et al (2020) dataset as selected by GeneSortR20. As each gene was simulated independently, it should in theory retain the topology of that initial single gene dataset, thereby replicating the discordance present in the original dataset. Furthermore, by directly comparing our random samples of simulated datasets to the most informative genes of the source dataset, this should disadvantage Scoutknife, as some of the simulated data may support a separate alternative topology to either the single gene or the real informative gene topology.系统发育自举法（phylogenetic bootstrap）由Felsenstein于1985年首次提出，是评估系统发育数据集稳健性的关键统计方法。其核心概念为伪采样（pseudosampling）：通过从构建系统发育所用的初始数据集生成新的复本，以此对数据进行评估。借此，系统发育支持度指标可弥补完美无限数据集的缺失。不过，若拥有无限数据集，便可直接从中抽取较小的复本，在同一次分析中同时获得系统发育树及其统计稳健性。 随着全基因组测序技术的发展，我们的数据集规模与广度均得到极大拓展，且这一趋势还将持续。当数据集达到基因组级别的数千个基因时，我们如今可获取无限数据集的替代方案。据此，我们或可摒弃伪采样的思路，转而从分析所用的数千个基因中随机抽取小部分基因子集。 本文介绍了Scoutknife，一款刀切法（jackknife）风格的子采样实现工具，它通过从初始大基因数据集随机抽取少量基因，生成100个数据集，以此共同构建系统发育假设并评估其稳健性。我们利用18个已发表的数据集与100项模拟研究，证明Scoutknife具有保守性，且能清晰反映全基因组范围内的冲突与不一致性，无需基于传统模型选择标准进行子采样。 方法 数据集构建与分析 无论是真实数据分析还是模拟数据分析（详细说明见下文），我们均从源数据集中随机选取100次100个基因，借助Scoutknife工具包（https://github.com/JFFleming/Scoutknife）生成100个100基因串联样本数据集。Scoutknife脚本包需预先安装catsequences工具，该工具可从https://github.com/ChrisCreevey/catsequences获取。 每个Scoutknife数据集的系统发育树均通过IQ-TREE v1.6.1224构建，采用Modeltest25工具，为每个基因分配独立的进化模型，且不进行分区合并。作为一种基于数据密度的技术，Scoutknife在高数据密度场景下表现更佳，此时分区可顺利合并。因此，本研究刻意限制了Scoutknife的效能，以在正常条件以外的、拟合模型差异更大的场景下测试其性能，同时考虑到测试数据集与模拟样本数量庞大，可节省计算资源。为进一步推动分析的并行化，我们使用Scoutknifette（https://github.com/Togtja/scoutknifette）提交数据集的系统发育分析任务。Scoutknifette是一款针对群组通讯服务Discord的自定义高性能计算（HPC）Webhook工具，可轻松修改以适配任何需要多批次提交与队列追踪的HPC任务。 随后，将每个Scoutknife样本数据集生成的树合并为单个树列表文件（详见补充材料），并利用Phylobayes26中的bpcomp工具构建共识树，设置预烧期为0、采样率为1，对树列表中的每棵树进行采样。我们分别构建了70%严格共识树与50%多数共识树，并对结果进行比较。在两个案例（蜘蛛目Araneae与鳞翅目Lepidoptera）中，我们采用相同方法构建了30%相对多数共识树，以进一步探索数据，具体详见结果与讨论部分。在单个案例（辐鳍鱼纲Actinopterygii）中，两个物种的占据率极低（在全部1105个基因的数据集中，海鳗Muraenesox cinerus仅含1个基因，鲭Scomber scombrus仅含15个基因），导致多数Scoutknife样本未包含这两个类群的代表。为解决这一问题，我们使用了DendroPy包27中的sumtrees.py v4.5.2工具，因其可从包含可变类群数量的树列表中构建共识树。 我们参照由GeneSortR20筛选出的该数据集250个最具信息性基因的已发表拓扑结构，计算了四分体相似度（Quartet Similarity）、四分体差异度（Quartet Divergence）、节点冲突度（Node Conflict）、节点一致性（Node Agreement）、严格联合断言（Strict Joint Assertions）、半严格联合断言（Semi-Strict Joint Assertions）、对称差（Symmetric Difference）、Marczewski-Steinhaus距离、Steel-Penny指标与整体相似度（Overall Similarity）。对于模拟数据集，我们采用由GeneSortR20筛选出的Milla等人2020年28数据集的250个最具信息性基因的分析结果所得到的拓扑结构。上述相似度指标通过R语言29中的‘Quartet’库计算得到。 真实测试数据集 为评估Scoutknife的效能，我们分析了18个真实数据集28,30-46，即此前GeneSortR20同类基准研究所用的数据集。这些数据集的基因数介于1049至5105之间，类群数介于30至332之间，涵盖动物、植物与真菌类群（见表1）。与此前研究不同，我们未移除占据率低于50%的基因：由于Scoutknife依赖数据密度，在低占据率场景下性能或会下降，因此此举可更清晰地展现该方法在各类真实数据集上的表现。随后，我们将所得树拓扑结构与通过GeneSortR20筛选出的250个最具信息性基因的分析结果所得到的拓扑结构进行比较，以验证Scoutknife共识树是否得到了一致的拓扑假设。 模拟数据集 为进一步评估Scoutknife的性能，我们利用IQ-TREE v2.2.047,48中实现的AliSim比对模拟功能，生成了100个模拟数据集。具体而言，我们针对Milla等人2020年28发表的Heliozelidae数据集（该数据集在我们的真实数据研究中属于小型数据集，仅含1049个基因）中的每个基因，独立生成100次模拟。该数据集对Scoutknife而言颇具挑战性，且AliSim的比对模拟功能47可生成贴合真实基因的比对数据集，包括低占据率与合理的比对长度变异。AliSim的执行命令如下： iqtree2 –alisim < Output > -s <Gene> --num-alignments 100 针对每组1049个模拟基因，我们构建了100个含100个基因的Scoutknife数据集，并采用与真实数据集一致的流程，利用IQ-TREE进行分析。随后，我们参照由GeneSortR20筛选出的Milla等人2020年数据集的250个最具信息性基因的分析结果所得到的已发表拓扑结构，计算了四分体相似度（Quartet Similarity）、四分体差异度（Quartet Divergence）、节点冲突度（Node Conflict）、节点一致性（Node Agreement）、严格联合断言（Strict Joint Assertions）、半严格联合断言（Semi-Strict Joint Assertions）、对称差（Symmetric Difference）、Marczewski-Steinhaus距离、Steel-Penny指标与整体相似度（Overall Similarity）。由于每个基因均为独立模拟，理论上可保留初始单基因数据集的拓扑结构，从而复刻原始数据集中存在的不一致性。此外，通过将我们的随机模拟数据集样本与源数据集的最具信息性基因直接比较，此举或会对Scoutknife不利，因为部分模拟数据可能支持与单基因拓扑结构或真实信息基因拓扑结构不同的替代拓扑结构。