Towards a fully automated underwater census for fish assemblages in the Mediterranean Sea

Assessing underwater biodiversity is labour-intensive and costly, but is crucial for measuring the extent of the decline in local fish stock. In most cases, Underwater Visual Census (UVC) is the preferred method, however this can be costly in terms of human effort and is limited by meteorological and logistical factors. Advances in technology allows the utilisation of more autonomous video recording methods (i.e. Remote Operated Vehicles (ROV)) which addresses these limitations. This study used a transect-wise UVC coupled with diver operated videos (DOV). For the video analysis, a comprehensive fully automated pipeline was developed to extract frames from DOV and perform colour correction. This pipeline integrates a YOLO-based model to detect 20 Mediterranean fish species and validate the presence or absence of each species within individual transects. This study was conducted to evaluate the feasibility of using video-based methods for UVC with minimal human-input. The result of automated video analysis were in agreement with manual video counting, validating the autonomous and bias-free procedure for video assessment. In conclusion, utilising a minimal-human-input video method liberates the data acquisition from limiting factors (i.e. meteorological and logistical) and automation of this video analysis significantly reduces the labour and time required. For future fieldwork campaigns, the video data collection protocol needs to be modified to better resemble traditional UVC and enhance this acquisition method.

Methods
1. Study area and data collection

The training dataset (DATAT ) was gathered in eight different locations in the Mediterranean Sea along the French Riviera, following the same UVC protocol on each site (Harmelin-Vivien et al., 1985). The depth ranged from 1-37m and was carried out during the whole year in 2022 (cold and warm season) to cover the full range of conditions and possibilities of fish occurrences.

The experimental dataset (DATAE) was recorded in October 2023 in and around two protected areas, one no-take zone (Cap Roux) and one Natura2000 site (Corniche Varoise), which both have elevated biodiversity. The specific coordinates and meta data can be found in the supplementary material (Table S1). A total of 64 videos, each corresponding to a transect, from 14 sites (8 on seagrass meadows and 6 on rocky substrates) were evaluated and compared. Each site consists of 3 to 6 transects, depending on the availability of video recordings and UVC data from the divers.

The videos were obtained with GoPro HERO 9 cameras, mounted on the clipboards (Fig. 1) used by the divers to note the number of fish per species with their respective size category (variable number of categories per species). The videos were recorded with a framerate of 24 frames per second (FPS) and full high definition resolution (1920x1080px). Frames were extracted from these recordings with a framerate of 1 FPS for DATAT and 5 FPS for DATAE. Fish visible for less than 1 second (less than 5 frames) in the videos of DATAE will not be considered in the methodology evaluation as they were unlikely to be actual detections.

2. Image preprocessing

The frames were processed in a next step by a marine biology expert to guarantee correctly identified species in the videos. To ensure a good species coverage, 19 different species and an ’Other’ class were labelled manually in the frames resulting in 13,033 images (131 videos in the training set and 47 independent videos in the test set) in DATAT with a total of 68,573 (train = 40,379, test = 28,194) individual fish labels (species breakdown in Table S3) and 8,739 miscellaneous labels such as background and diver. The ’Other’ class includes species (Table S2) that have insufficient occurrences in the test videos (n < 100). Since there was a wide range of conditions in the videos, a preprocessing was applied to both datasets to enhance each image colour range. For this purpose, a pretrained UIEC2-Net model (Y. Wang et al., 2021) was utilised to enhance the images. As the last preprocessing step, images were rescaled to 960x960px.