Labeled Fishes in the Wild 鱼类标注数据集

Labeled Fishes in the Wild 为鱼类图像数据集，图像中包含鱼类、无脊椎动物和河床，通过部署在远程操作潜水器上的渔业统计摄像系统拍摄得到的。鱼类位置数据被包括在相应的数据文件中（ dat，vec 和 info），标注了鱼在图像中的位置。 该野生鱼类标记图像数据集是由NOAA渔业（国家海洋渔业服务机构），以鼓励开发，测试，以及为不受约束的水下图像自动图像分析算法的性能评估。 数据集包括鱼，无脊椎动物和海床的图像，这些图像是使用部署在遥控车（ROV）上的摄像系统采集的，用于渔业调查。注释数据包含在随附的数据文件（.dat，.vec和.info）中，这些文件描述了图像中标记的鱼目标的位置。 该手稿（Cutter等人，2015年）演示了基于使用训练图像数据集开发并使用测试集进行评估的分类器自动检测鱼类的方法。该数据集可用于进一步开发复杂环境中鱼类或无脊椎动物的检测；跟踪视频图像序列中的多个动物目标；动物种类的识别和分类；立体图像对中动物的测量；和海底栖息地的特征。 推荐引用：G. Cutter；K. Stierhoff；Zeng，J.（2015）“使用Haar级联和新的图像数据集在不受约束的水下视频中自动检测石鱼：带标签的野外鱼类，” IEEE计算机视觉应用冬季研讨会，第57-62页。 负责这些数据的NOAA科学家可能拥有图像档案，这些档案可以为协作应用和评估算法提供额外的机会。应该在出版物中提供使用这些数据集的信誉，如数据集存档中包含的“ how-to-cite.txt”文档中所述或如上所述。 野生图像数据集（v.1.1）中带有标签的鱼。 带有 标签的野生鱼具有三个组成部分：训练和验证的正图像集（已验证的鱼），负图像集（非鱼）和测试图像集。训练和测试集具有随附的注释数据，这些数据定义了图像中每个标记的鱼目标对象的位置和范围。这些代表专家分析人员定义的边界矩形，格式为OpenCV使用的.dat文件。 培训和验证正面图像集：包含石斑鱼（Sebastes spp。）和海床附近的其他相关物种的图像，这些图像是使用部署在遥控车辆（ROV）上的前斜向数码相机拍摄的由西南渔业科学中心在对加利福尼亚南部沿海的岩石海底环境进行调查期间。这些摄像机的静止帧代表ROV缓慢移动且运动影响不是影响因素的调查期间的实例。训练集包含929个图像文件，其中包含1005个带有相关注释的标记鱼（它们的标记位置和边界矩形）。这些标记定义了相机的各种物种，大小和范围的鱼，并且包括背景组成不同的部分。 培训和验证负面图片集：包括3167张图片。在野外训练和测试图像集中，从带标签的鱼中提取了可下载档案中提供的147个海底负图像（提取了不含鱼的区域）。其余3020张图像可从 OpenCV HaarTraining上的教程中获得，也可从 data negatives目录中获得。 测试图像集：包含在近海底鱼类调查期间使用ROV的高清（HD; 1080i）摄像机收集的图像序列。用于检测的测试图像包括来自ROV调查的视频镜头。用于评估本研究检测器的视频片段（“ TEST_VIDEO_ROV10.mp4”； 210帧，每秒3帧（fps））代表原始视频序列的第10帧（2分钟，约30 fps）。对于210帧3fps测试视频，所有鱼目标均带有注释。测试视频中鱼的注释包括“已验证”或“明显”的描述符，其中已验证表明视频分析人员可以识别出鱼，并且明显的物体被认为是鱼，但无法根据单个框架中可见的属性进行验证。这些明显的鱼在远处看起来像淡淡的斑点。这些区别是在注释数据中进行的，因为我们认为某些分类器会检测到这些明显的鱼，但我们不希望分类器能够这样做。我们也不一定要探测器这样做。也就是说，如果分类器正在检测那些明显的鱼类，那么它可能正在检测图像中的许多其他非鱼类目标，从而使其效率低下且不切实际。在数据集测试视频的带注释的帧中总共标记了2061个鱼对象。其中，有1008条是经过验证的鱼，而1053条是明显的鱼。在序列中，ROV正在移动；背景似乎正在移动并且从不同方向被照亮（随着ROV的移动和旋转）；水流中的小颗粒流过去；鱼静止或以各种速度运动；鱼有很多方向。一些鱼被部分地藏在岩石或缝隙中；远处出现一些模糊的鱼状物体。 野生数据集（v1.0，2014年12月）中的原始Labeled鱼类仅包含抽取后的测试视频序列（“ Test_ROV_video_h264_decim.mp4”），该序列仅包含原始视频中的标记帧。完整帧率视频的十分之一帧被标记为鱼类目标的位置。此版本的数据集（v1.1，2015年1月）还包含完整的测试视频序列（“ Test_ROV_video_h264_full.mp4”）。完整视频和抽取后的视频都带有附带分析标记的文本文件（遵循OpenCV .dat文件约定）。通常，对于m个标记，格式为：视频文件名（帧号）标记数x1 y1 w1 h1 x2 y2 w2 h2 ... xm ym wm hm。例如，如果有两个标记，最后的8个值定义边界矩形：Test_ROV_video_h264_full.mp4（fr_14）2 1021 362 94 63 953 289 9061。抽取视频的标记文件（“ Test_ROV_video_h264_decim_marks.dat”）指示抽取和完整序列的帧号，例如Test_ROV_video_h264_decim.mp4（fr_1）（fullfr_14）2 1021 362 94 63 953 289 9061。完整视频中有2101帧，抽取后的视频中有210帧，但是标记了206帧；即，一些检查的框架不包含鱼。）表示抽取和完整序列的帧号，例如Test_ROV_video_h264_decim.mp4（fr_1）（fullfr_14）2 1021 362 94 63 953 289 9061。抽取后的视频中有2101帧，抽取后的视频中有210帧，但是206标有框架；即，一些检查的框架不包含鱼。）表示抽取和完整序列的帧号，例如Test_ROV_video_h264_decim.mp4（fr_1）（fullfr_14）2 1021 362 94 63 953 289 9061。抽取后的视频中有2101帧，抽取后的视频中有210帧，但是206标有框架；即，一些检查的框架不包含鱼。

Labeled Fishes in the Wild is a fish image dataset containing fish, invertebrates and seabeds, captured by a fisheries statistical camera system mounted on a remotely operated underwater vehicle (ROV). Fish location data annotating the positions of fish in the images is included in the corresponding .dat, .vec and .info files.

This labeled wild fish image dataset was developed by NOAA Fisheries (National Marine Fisheries Service) to encourage the development, testing, and performance evaluation of unconstrained underwater image automatic analysis algorithms. The dataset comprises images of fish, invertebrates and seabeds collected by a camera system mounted on an ROV for fisheries surveys, with annotation data in the accompanying .dat, .vec and .info files describing the positions of marked fish targets in the images.

This manuscript (Cutter et al., 2015) demonstrates an automatic fish detection approach using a classifier trained on the dataset's training images and evaluated with the test set. This dataset supports further development of fish or invertebrate detection in complex environments, tracking multiple animal targets in video sequences, species identification and classification, animal measurement in stereo image pairs, and seabed habitat characterization.

Recommended citation: G. Cutter; K. Stierhoff; Zeng, J. (2015) "Automatic Detection of Rockfish in Unconstrained Underwater Video Using Haar Cascades and a New Image Dataset: Labeled Fishes in the Wild", IEEE Winter Conference on Applications of Computer Vision, pp. 57-62.

NOAA scientists responsible for this dataset may hold additional image archives enabling further collaborative applications and algorithm evaluations. Credit for using the dataset should be provided in publications as described in the "how-to-cite.txt" document included with the dataset archive or as noted above.

Labeled Fishes in the Wild (v.1.1) includes three components: a positive image set for training and validation (verified fish), a negative image set (non-fish), and a test image set. The training and test sets come with annotation data defining the positions and extents of each marked fish target, which are bounding boxes defined by expert analysts in the format used by OpenCV for .dat files.

### Training and Validation Positive Image Set
Contains images of rockfish (Sebastes spp.) and other related species near rocky seabeds, captured by a forward-looking oblique digital camera mounted on an ROV during surveys of southern California's coastal rocky benthic environments by the Southwest Fisheries Science Center. Still frames were collected during slow ROV transits with negligible motion artifacts. The training set includes 929 image files with 1005 annotated labeled fish, each with their marker positions and bounding boxes. These annotations cover fish of varying species, sizes, and extents across diverse background compositions.

### Training and Validation Negative Image Set
Includes 3167 images: 147 seabed negative images (extracted as fish-free regions from the labeled training and test sets) are provided in the downloadable archive, while the remaining 3020 images are available from the OpenCV HaarTraining tutorial and the `data negatives` directory.

### Test Image Set
Comprises image sequences collected with an HD (1080i) camera mounted on an ROV during near-seabed fish surveys. Test images include footage from ROV surveys. The video clip used to evaluate the study's detector ("TEST_VIDEO_ROV10.mp4") has 210 frames at 3 fps, derived from a 2-minute segment (~30 fps original frame rate) of the full video sequence.

All fish targets in the 210-frame 3 fps test video are annotated. Annotations use two descriptors: "verified" for fish identifiable by video analysts, and "distinct" for objects presumed to be fish but unverifiable from single-frame visible attributes. Distinct fish appear as faint distant specks. This distinction is included in annotations because some classifiers may detect these distinct fish, though this is not required; detecting such indistinct targets would likely also capture many non-fish objects, making the detector inefficient and impractical. A total of 2061 fish objects are annotated across the test video's labeled frames: 1008 verified fish and 1053 distinct fish.

In the test sequence, the ROV is moving, causing background shift and variable directional lighting as the platform translates and rotates. Small particles flow in the water current, fish remain stationary or move at varying speeds, and fish are oriented in multiple directions. Some fish are partially hidden behind rocks or in crevices, with indistinct fish-like objects appearing at distance.

The original Labeled Fishes in the Wild dataset (v1.0, December 2014) only included the downsampled test video sequence ("Test_ROV_video_h264_decim.mp4"), which contained only labeled frames from the original video, with 1 out of every 10 full-frame-rate video frames annotated for fish targets. This v1.1 (January 2015) release also includes the full test video sequence ("Test_ROV_video_h264_full.mp4"). Both full and downsampled videos have accompanying annotation text files following the OpenCV .dat file convention.

The standard annotation format for m targets is: [video filename] ([frame number]) [number of annotations] x1 y1 w1 h1 x2 y2 w2 h2 ... xm ym wm hm. For example, two annotations would appear as: Test_ROV_video_h264_full.mp4 (fr_14) 2 1021 362 94 63 953 289 9061. The downsampled video's annotation file ("Test_ROV_video_h264_decim_marks.dat") lists both downsampled and full sequence frame numbers, e.g., Test_ROV_video_h264_decim.mp4 (fr_1) (fullfr_14) 2 1021 362 94 63 953 289 9061. The full video contains 2101 frames, the downsampled video contains 210 frames, of which 206 are annotated — some examined frames contain no fish.

搜集汇总

背景与挑战

背景概述

该数据集是一个包含鱼类、无脊椎动物和河床图像的标注数据集，专为开发自动图像分析算法设计。图像通过远程操作潜水器拍摄，包含详细的鱼类位置标注，适用于鱼类检测、跟踪和分类等研究。