Estimating ancestral states of complex characters: A case study on the evolution of feathers

Feathers are a key novelty underpinning the evolutionary success of birds,
yet the origin of feathers remains poorly understood. Debates about
feather evolution hinge upon whether filamentous integument has evolved
once or multiple times independently in the lineage leading to modern
birds. These contradictory results stem from methodological differences in
statistical ancestral state estimates. Here, we conduct a comprehensive
comparison of ancestral state estimation methodologies applied to
stem-group birds, testing the role of outgroup inclusion, tree time
scaling method, model choice, and character coding strategy. Models are
compared based on their Akaike Information Criteria (AIC), mutual
information, as well as the uncertainty of marginal ancestral state
estimates. Our results demonstrate that ancestral state estimates of
stem-bird integument are strongly influenced by tree time scaling method,
outgroup selection, and model choice, while character coding strategy
seems to have less effect on the ancestral estimates produced. We identify
the best-fitting and most generalizable models using AIC scores and
leave-one-out cross-validation (LOOCV), respectively. Our analyses broadly
support the independent origin of filamentous integument in dinosaurs and
pterosaurs and support a younger evolutionary origin of feathers than has
been suggested previously. In terms of model selection, we observe little
correlation between AIC/AICc and LOOCV error, suggesting that, for our
dataset, model fit does not reliably predict generalizability. However,
both approaches favor models that infer a similar pattern of feather
evolution. More globally, our study highlights that special care must be
taken in selecting the outgroup, tree, and model when conducting ASE
analyses.