Black-grass monitoring using hyperspectral image data is limited by between-site variability

Many important ecological processes play out over large geographic ranges, and accurate large-scale monitoring of populations is a requirement for their effective management. Of particular interest are agricultural weeds which cause widespread economic and ecological damage. However, the scale of weed population data collection is limited by an inevitable trade-off between quantity and quality. Remote sensing offers a promising route to large-scale collection of population state data. However, a key challenge is to collect high enough resolution data and account for between-site variability in environmental (i.e. radiometric) (Peleg, Andersen, & Yang 2005, Nansen, Mantri, & Lee, 2023) conditions that may make prediction of population states in new data challenging. Here we use a multi-site hyperspectral image data set in conjunction with ensemble learning techniques in an attempt to predict densities of an arable weed (Alopecurus myosuroides) across an agricultural landscape. We..., Data consists of categorical assessments of weed (black-grass) density from 31 fields across the UK. Each field is divided up into 20x20m quadrats and assigned one of 3 density states (low, medium, high) as in Queenborough et al (2011). For each quadrat we provide 1000 pixel level samples of 120 spectral bands (hyperspectral image data used in the manuscript), as well as 14 vegetation indeces derived from these bands. Code is included to fit an ensemble of random forests to these data to attempt to predict the quadrat-level density states from hyperspectral (HS) and vegetation index (VI) data. Cross validation code is included to assess whether out-of-sample (i.e. new fields) predictive performance can be increased via ensemble models. Ensembles are weighted towards fields with more similar spectral similarity to the out-of sample data. , , # Data from: Black-grass monitoring using hyperspectral image data is limited by between-site variability

[https://doi.org/10.5061/dryad.qz612jmqp](https://doi.org/10.5061/dryad.qz612jmqp)

## Description of the data and file structure

### Data:

Contains data files to replicate analyses:

**multi_state_fields.rds** - index of fields containing more than one density state.
**opt_hypars.rds** - contains values of optimal random forest hyperparameters - please see the supplementary material from the paper for an explanation.
**sub_samp_HS_data.rds** - contains training data for raw hyperspectral bands.
**sub_samp_VGI_data.rds** - contains training data for derived vegetation indeces.

Within the last two files the data have the following columns:

DS - the density state of the quadrat.
image_strip - an index of the combined images used to derive the hyperspectral or vegetation index data.
grid - a quadrat ID index.
x - an X coordinate from the given field.
y - a Y coordinate from the gi...,