Quantifying individual variation in reaction norms: mind the residual

Phenotypic plasticity is a central topic in ecology and evolution.
Individuals may differ in the degree of plasticity
(individual-by-environment interaction (I×E)), which has implications for
the capacity of populations to respond to selection. Random regression
models (RRMs) are a popular tool to study I×E in behavioural or
life-history traits, yet evidence for I×E is mixed, differing between
species, populations, and even between studies on the same population. One
important source of discrepancies between studies is the treatment of
heterogeneity in residual variance (heteroscedasticity). To date, there
seems to be no collective awareness among ecologists of its influence on
the estimation of I×E or a consensus on how to best model it. We performed
RRMs with differing residual variance structures on simulated data with
varying degrees of heteroscedasticity and plasticity, sample size and
environmental variability to test how RRMs would perform under each
scenario. The residual structure in the RRMs affected the precision of
estimates of simulated I×E as well as statistical power, with substantial
lack of precision and high false-positive rates when sample size,
environmental variability and plasticity were small. We show that model
comparison using information criteria can be used to choose among residual
structures, and reinforce this point by analysis of real data of two study
populations of great tits (Parus major). We provide guidelines that can be
used by biologists studying I×E that, ultimately, should lead to a
reduction in bias in the literature concerning the statistical evidence
and the reported magnitude of variation in plasticity.