A new pipeline for removing paralogs in target enrichment data

Target enrichment (such as Hyb-Seq) is a well-established high throughput
sequencing method that has been increasingly used for phylogenomic
studies. Unfortunately, current widely used pipelines for analysis of
target enrichment data do not have a vigorous procedure to remove paralogs
in target enrichment data. In this study, we develop a pipeline we call
Putative Paralogs Detection (PPD) to better address putative paralogs from
enrichment data. The new pipeline is an add-on to the existing HybPiper
pipeline, and the entire pipeline applies criteria in both sequence
similarity and heterozygous sites at each locus in the identification of
paralogs. Users may adjust the thresholds of sequence identity and
heterozygous sites to identify and remove paralogs according to the level
of phylogenetic divergence of their group of interest. The new pipeline
also removes highly polymorphic sites attributed to errors in sequence
assembly and gappy regions in the alignment. We demonstrated the value of
the new pipeline using empirical data generated from Hyb-Seq and the
Angiosperm 353 kit for two woody genera Castanea (Fagaceae, Fagales) and
Hamamelis (Hamamelidaceae, Saxifragales). Comparisons of datasets showed
that the PPD identified many more putative paralogs than the popular
method HybPiper. Comparisons of tree topologies and divergence times
showed evident differences between data from HybPiper and data from our
new PPD pipeline. We further evaluated the accuracy and error rates of PPD
by BLAST mapping of putative paralogous and orthologous sequences to a
reference genome sequence of Castanea mollissima. Compared to HybPiper
alone, PPD identified substantially more paralogous gene sequences that
mapped to multiple regions of the reference genome (31 genes for PPD
compared with 4 genes for HybPiper alone). In conjunction with HybPiper,
paralogous genes identified by both pipelines can be removed resulting in
the construction of more robust orthologous gene datasets for phylogenomic
and divergence time analyses. Our study demonstrates the value of Hyb-Seq
with data derived from the Angiosperm 353 probe set for elucidating
species relationships within a genus, and argues for the importance of
additional steps to filter paralogous genes and poorly aligned regions
(e.g., as occur through assembly errors), such as our new PPD pipeline
described in this study.