CVB: A Video Dataset of Cattle Visual Behaviors

Existing image/video datasets for cattle behavior recognition are mostly small, lack well-defined labels, or are collected in unrealistic controlled environments. This limits the utility of machine learning (ML) models learned from them. Therefore, we introduce a new dataset, called Cattle Visual Behaviors (CVB), that consists of 502 video clips, each fifteen seconds long, captured in natural lighting conditions, and annotated with eleven visually perceptible behaviors of grazing cattle. By creating and sharing CVB, our aim is to develop improved models capable of recognizing all important behaviors accurately and to assist other researchers and practitioners in developing and evaluating new ML models for cattle behavior classification using video data.\nThe dataset is presented in the form of following three sub-directories.\n1. raw_frames: contains 450 frames in each sub folder, representing 15 sec video, taking at a frames rate of 30 FPS,\n2. annotations: contains the json files corresponding to the raw_frames folder. We have one json file for one video, containing the bounding box annotations for each cattle and their associated behaviors, and\n3. CVB_in_AVA_format: contains the CVB data in the standard AVA dataset format which we have used to apply SlowFast model.\nLineage: We use the Computer Vision Annotation Tool (CVAT) to collect our annotations. To make the procedure more efficient, we perform an initial detection and tracking of cattle in the videos using appropriate pre-trained models. The results are corrected by domain experts along with cattle behavior labeling in CVAT. The pre-hoc detection and tracking step significantly reduces the manual annotation time and effort.现有的用于牛行为识别的图像/视频数据集大多规模偏小、标签定义模糊，或是在脱离实际的受控环境中采集，这极大限制了基于此类数据集训练的机器学习（Machine Learning, ML）模型的实用价值。为此，我们推出了一款全新数据集——牛视觉行为数据集（Cattle Visual Behaviors，CVB），该数据集包含502段时长为15秒的视频片段，均采集于自然光照环境，并标注了11种可通过视觉识别的牛行为。我们创建并共享CVB的目标，是开发出能够精准识别所有关键行为的优化模型，同时助力其他研究人员与从业者开发、评估基于视频数据的牛行为分类新型机器学习模型。 本数据集包含以下三个子目录： 1. raw_frames（原始帧）：每个子文件夹包含450帧图像，对应一段15秒的视频，采集帧率为30 FPS（帧每秒）； 2. annotations（标注文件）：包含与raw_frames文件夹对应的JSON格式文件，每段视频对应一个JSON文件，内含每头牛的边界框标注及其关联的行为信息； 3. CVB_in_AVA_format（AVA格式CVB数据集）：包含采用标准AVA数据集格式存储的CVB数据，供SlowFast模型使用。 数据集溯源：我们采用计算机视觉标注工具（Computer Vision Annotation Tool，CVAT）完成标注工作。为提升标注效率，我们先借助适配的预训练模型对视频中的牛进行初步检测与跟踪，随后由领域专家在CVAT平台中对检测跟踪结果进行修正，并同步完成牛行为标注。该前置检测与跟踪步骤显著降低了手动标注的时间与人力投入。