What is our power to detect device effects in animal tracking studies?

The use of bio-logging devices to track animal movement continues to grow
as technological advances and device miniaturisation allow researchers to
study animal behaviour in unprecedented detail. Balanced against the
remarkable data that bio-loggers can provide is a need to understand the
impact of devices on animal behaviour and welfare. Recent meta-analyses
have demonstrated impacts of device attachment on animal behaviour, but
there is concern about the frequency and clarity with which device effects
are reported. One aspect lacking in many studies is assessment of the
statistical power of tests of device effects, yet such information would
assist the interpretation of results. We address this issue by providing
an overview of the statistical power, as well as the Type M (magnitude)
and Type S (sign) error rate, of tests of device effects within the avian
tracking literature across a range of assumed effect sizes. The median
power of statistical tests ranged from 9% to 65% across a range of assumed
effect sizes corresponding to benchmark values for small, moderate and
large effects (d = 0.2, 0.5, 0.8 respectively). Moreover, when using
effect sizes derived from previous a meta-analysis (d = 0.1) median power
was only 6%.  When assuming smaller effect sizes, statistical
tests were characterised by high Type M and Type S error rates, suggesting
that statistically significant results of device effects will tend to
exaggerate the size of such effects and may estimate the sign of an effect
in the wrong direction. Well-designed tracking studies will reduce device
effects to low levels and consequently issues associated with low power
will be commonplace. Nevertheless, assessment of device effects remains
important, particularly when embarking on novel tracking studies. We
recommend that statistical tests of device effects are reported clearly
and are routinely accompanied by assessment of statistical power,
including Type M and Type S errors, based upon realistic external
estimates of effect size. Reporting the statistical power can help avoid
the pitfalls of overstating results from individual studies, shift the
emphasis to accurate reporting of effect sizes and guide decisions about
the ethical impacts of device attachment.