Simulation dataset for Clustering-based loci subsampling helps for robust phylogenomics

Phylogenomic subsampling helps to mitigate data heterogeneity or reduce computational burden by selecting a subset of loci from genome-scale datasets for phylogenetic analysis. This approach typically involves filtering loci based on various properties of loci but lacks a consensus in practice. We proposed a novel multivariate, clustering-based approach that combines non-linear manifold learning UMAP dimension reduction with hierarchical clustering to identify and sort clusters based on their metrics possessing phylogenetic signal. Testing across 60 simulated and 16 empirical datasets, we evaluated its performance under varying data conditions including data source, subsampling size, species tree methods, and levels of incomplete lineage sorting (ILS). Combined summary statistics from both alignments and gene trees were preferred for clustering analyses, while statistics derived solely from gene trees (edge branch lengths and supports) were found to be less reliable, particularly at higher ILS levels. Subsampling strategies based on summary statistics or gene—species tree discordance consistently outperformed alternative approaches across all datasets. In empirical datasets, our method improved species tree accuracy even when filtering substantial portions of loci, with minimal adverse effects. Our clustering-based approach effectively identifies and retains loci with strong phylogenetic signals while excluding problematic sequences, demonstrating the potential for enhancing phylogenetic inference accuracy.