Data from: Towards automated annotation of benthic survey images: variability of human experts and operational modes of automation

Global climate change and other anthropogenic stressors have heightened
the need to rapidly characterize ecological changes in marine benthic
communities across large scales. Digital photography enables rapid
collection of survey images to meet this need, but the subsequent image
annotation is typically a time consuming, manual task. We investigated the
feasibility of using automated point-annotation to expedite cover
estimation of the 17 dominant benthic categories from survey-images
captured at four Pacific coral reefs. Inter- and intra- annotator
variability among six human experts was quantified and compared to semi-
and fully- automated annotation methods, which are made available at
coralnet.ucsd.edu. Our results indicate high expert agreement for
identification of coral genera, but lower agreement for algal functional
groups, in particular between turf algae and crustose coralline algae.
This indicates the need for unequivocal definitions of algal groups,
careful training of multiple annotators, and enhanced imaging technology.
Semi-automated annotation, where 50% of the annotation decisions were
performed automatically, yielded cover estimate errors comparable to those
of the human experts. Furthermore, fully-automated annotation yielded
rapid, unbiased cover estimates but with increased variance. These results
show that automated annotation can increase spatial coverage and decrease
time and financial outlay for image-based reef surveys.