EOAD (Egocentric Outdoor Activity Dataset)

EOAD is a collection of videos captured by wearable cameras, mostly of sports activities. It contains both visual and audio modalities. It was initiated by the HUJI and FPVSum egocentric activity datasets. However, the number of samples and diversity of activities for HUJI and FPVSum were insufficient. Therefore, we combined these datasets and populated them with new YouTube videos. The selection of videos was based on the following criteria: The videos should not include text overlays. The videos should contain natural sound (no external music) The actions in videos should be continuous (no cutting the scene or jumping in time) Video samples were trimmed depending on scene changes for long videos (such as driving, scuba diving, and cycling). As a result, a video may have several clips depicting egocentric actions. Hence, video clips were extracted from carefully defined time intervals within videos. The final dataset includes video clips with a single action and natural audio information. Statistics for EOAD: 30 activities 303 distinct videos 1392 video clips 2243 minutes labeled videos clips The detailed statistics for the selected datasets and the crawled videos clips from YouTube are given below: HUJI: 49 distinct videos - 148 video clips for 9 activities (driving, biking, motorcycle, walking, boxing, horse riding, running, skiing, stair climbing) FPVSum: 39 distinct videos - 124 video segments for 8 activities (biking, horse riding, skiing, longboarding, rock climbing, scuba, skateboarding, surfing) YouTube: 216 distinct videos - 1120 video clips for 27 activities (american football, basketball, bungee jumping, driving, go-kart, horse riding, ice hockey, jet ski, kayaking, kitesurfing, longboarding, motorcycle, paintball, paragliding, rafting, rock climbing, rowing, running, sailing, scuba diving, skateboarding, soccer, stair climbing, surfing, tennis, volleyball, walking) The video clips used for training, validation and test sets for each activity are listed in Table 1. Multiple video clips may belong to a single video because of trimming it for some reasons (i.e., scene cut, temporary overlayed text on videos, or video parts unrelated to activities). While splitting the dataset, the minimum number of videos for each activity was selected as 8. Additionally, the video samples were divided as 50%, 25%, and 25% for training (minimum four videos), validation (minimum two videos), and testing (minimum two videos), respectively. On the other hand, videos were split according to the raw video footage to prevent the mixing of similar video clips (having the same actors and scenes) into training, validation, and test sets. Therefore, we ensured that the video clips trimmed from the same videos were split together into training, validation, or test sets to satisfy a fair comparison. Some activities have continuity throughout the video, such as scuba, longboarding, or riding horse, which also have an equal number of video segments with the number of videos. However, some activities, such as skating, occurred in a short time, making the number of video segments higher than the others. As a result, the number of video clips for training, validation, and test sets was highly imbalanced for the selected activities (i.e., jet ski and rafting have 4; however, soccer has 99 video clips for training). Table 1 - Dataset splitting for EOAD Train Validation Test Action Label #Clips Total Duration #Clips Total Duration #Clips Total Duration AmericanFootball 34 00:06:09 36 00:05:03 9 00:01:20 Basketball 43 01:13:22 19 00:08:13 10 00:28:46 Biking 9 01:58:01 6 00:32:22 11 00:36:16 Boxing 7 00:24:54 11 00:14:14 5 00:17:30 BungeeJumping 7 00:02:22 4 00:01:36 4 00:01:31 Driving 19 00:37:23 9 00:24:46 9 00:29:23 GoKart 5 00:40:00 3 00:11:46 3 00:19:46 Horseback 5 01:15:14 5 01:02:26 2 00:20:38 IceHockey 52 00:19:22 46 00:20:34 10 00:36:59 Jetski 4 00:23:35 5 00:18:42 6 00:02:43 Kayaking 28 00:43:11 22 00:14:23 4 00:11:05 Kitesurfing 30 00:21:51 17 00:05:38 6 00:01:32 Longboarding 5 00:15:40 4 00:18:03 4 00:09:11 Motorcycle 20 00:49:38 21 00:13:53 8 00:20:30 Paintball 7 00:33:52 4 00:12:08 4 00:08:52 Paragliding 11 00:28:42 4 00:10:16 4 00:19:50 Rafting 4 00:15:41 3 00:07:27 3 00:06:13 RockClimbing 6 00:49:38 2 00:21:59 2 00:18:50 Rowing 5 00:47:05 3 00:13:21 3 00:03:26 Running 21 01:21:56 19 00:46:29 11 00:42:59 Sailing 7 00:39:30 4 00:14:39 6 00:15:43 Scuba 5 00:35:02 3 00:23:43 2 00:18:52 Skate 91 00:15:53 30 00:07:01 10 00:02:03 Ski 14 01:48:15 17 01:01:59 7 00:39:15 Soccer 102 00:48:39 52 00:13:17 16 00:06:54 StairClimbing 6 01:05:32 6 00:17:18 5 00:20:22 Surfing 23 00:12:51 17 00:06:52 10 00:07:04 Tennis 34 00:27:04 9 00:06:03 9 00:03:14 Volleyball 87 00:19:14 35 00:07:46 7 00:18:58 Walking 49 00:43:02 36 00:38:25 10 00:10:23 Total 30 740 20:22:37 452 09:20:23 200 08:00:08 EOAD Code Repository Scripts for downloading raw videos and trim them in to video clips are provided in this GitHub repository. Regarding the questions, please contact [e-mail].

EOAD是一款由穿戴式摄像机（wearable cameras）采集的视频数据集，内容以体育运动活动为主，涵盖视觉与音频两种模态。该数据集起源于HUJI与FPVSum两款自我中心视角活动数据集（egocentric activity datasets），但原数据集的样本数量与活动多样性均存在不足。因此，本研究团队整合了这两款数据集，并补充了来自YouTube的全新视频素材。

视频筛选遵循以下标准：视频不得包含文字叠加内容；需带有自然环境音（无外置背景音乐）；视频内的动作需保持连贯，无场景切换或时间跳切。针对驾驶、水肺潜水、骑行等长视频，我们将根据场景变化进行剪辑，因此单个原始视频可能包含多个自我中心视角动作片段。最终，我们从视频中精准划定的时间区间内提取视频片段，最终数据集包含仅含单一动作且带有自然音频的视频片段。

EOAD的统计信息如下：共涵盖30类活动，303个独立原始视频，1392个视频片段，总标注视频片段时长达2243分钟。各来源数据集的详细统计与YouTube爬取的视频片段信息如下：
HUJI数据集：49个独立原始视频，对应9类活动的148个视频片段，活动包括驾驶、骑行、摩托车运动、步行、拳击、骑马、跑步、滑雪、爬楼梯。
FPVSum数据集：39个独立原始视频，对应8类活动的124个视频片段，活动包括骑行、骑马、滑雪、长板运动、攀岩、水肺潜水、滑板运动、冲浪。
YouTube补充数据集：216个独立原始视频，对应27类活动的1120个视频片段，活动包括美式橄榄球、篮球、蹦极、驾驶、卡丁车、骑马、冰球、摩托艇、皮划艇、风筝冲浪、长板运动、摩托车运动、彩弹射击、滑翔伞、漂流、攀岩、划船、跑步、帆船运动、水肺潜水、滑板运动、足球、爬楼梯、冲浪、网球、排球、步行。

各活动的训练集、验证集与测试集的视频片段分布详见表1。由于剪辑原因（如场景切换、视频中临时叠加的文字、与活动无关的视频片段），多个视频片段可能来自同一个原始视频。在划分数据集时，我们将每类活动的最小原始视频数设为8。此外，数据集按照50%、25%、25%的比例划分为训练集（至少包含4个原始视频）、验证集（至少包含2个原始视频）与测试集（至少包含2个原始视频）。为避免来自同一演员与场景的相似视频片段被分散到不同集合中，我们按照原始视频素材进行划分，确保从同一原始视频剪辑得到的所有片段被统一划分到训练、验证或测试集中，以保证实验对比的公平性。

部分活动（如水肺潜水、长板运动、骑马）的视频片段数与原始视频数相等，这是因为这类活动在视频中具有全程连续性。而部分活动（如滑板）的单次动作时长较短，因此视频片段数多于其他活动。最终，训练、验证与测试集的视频片段数量存在显著不平衡：例如摩托艇与漂流的训练集片段数仅为4，而足球的训练集片段数高达99。

表1 - EOAD数据集划分
| 动作标签 | 训练集 | | 验证集 | | 测试集 | |
| --- | --- | --- | --- | --- | --- | --- |
| | 片段数 | 总时长 | 片段数 | 总时长 | 片段数 | 总时长 |
AmericanFootball | 34 | 00:06:09 | 36 | 00:05:03 | 9 | 00:01:20
Basketball | 43 | 01:13:22 | 19 | 00:08:13 | 10 | 00:28:46
Biking | 9 | 01:58:01 | 6 | 00:32:22 | 11 | 00:36:16
Boxing | 7 | 00:24:54 | 11 | 00:14:14 | 5 | 00:17:30
BungeeJumping | 7 | 00:02:22 | 4 | 00:01:36 | 4 | 00:01:31
Driving | 19 | 00:37:23 | 9 | 00:24:46 | 9 | 00:29:23
GoKart | 5 | 00:40:00 | 3 | 00:11:46 | 3 | 00:19:46
Horseback | 5 | 01:15:14 | 5 | 01:02:26 | 2 | 00:20:38
IceHockey | 52 | 00:19:22 | 46 | 00:20:34 | 10 | 00:36:59
Jetski | 4 | 00:23:35 | 5 | 00:18:42 | 6 | 00:02:43
Kayaking | 28 | 00:43:11 | 22 | 00:14:23 | 4 | 00:11:05
Kitesurfing | 30 | 00:21:51 | 17 | 00:05:38 | 6 | 00:01:32
Longboarding | 5 | 00:15:40 | 4 | 00:18:03 | 4 | 00:09:11
Motorcycle | 20 | 00:49:38 | 21 | 00:13:53 | 8 | 00:20:30
Paintball | 7 | 00:33:52 | 4 | 00:12:08 | 4 | 00:08:52
Paragliding | 11 | 00:28:42 | 4 | 00:10:16 | 4 | 00:19:50
Rafting | 4 | 00:15:41 | 3 | 00:07:27 | 3 | 00:06:13
RockClimbing | 6 | 00:49:38 | 2 | 00:21:59 | 2 | 00:18:50
Rowing | 5 | 00:47:05 | 3 | 00:13:21 | 3 | 00:03:26
Running | 21 | 01:21:56 | 19 | 00:46:29 | 11 | 00:42:59
Sailing | 7 | 00:39:30 | 4 | 00:14:39 | 6 | 00:15:43
Scuba | 5 | 00:35:02 | 3 | 00:23:43 | 2 | 00:18:52
Skate | 91 | 00:15:53 | 30 | 00:07:01 | 10 | 00:02:03
Ski | 14 | 01:48:15 | 17 | 01:01:59 | 7 | 00:39:15
Soccer | 102 | 00:48:39 | 52 | 00:13:17 | 16 | 00:06:54
StairClimbing | 6 | 01:05:32 | 6 | 00:17:18 | 5 | 00:20:22
Surfing | 23 | 00:12:51 | 17 | 00:06:52 | 10 | 00:07:04
Tennis | 34 | 00:27:04 | 9 | 00:06:03 | 9 | 00:03:14
Volleyball | 87 | 00:19:14 | 35 | 00:07:46 | 7 | 00:18:58
Walking | 49 | 00:43:02 | 36 | 00:38:25 | 10 | 00:10:23
总计 | 740 | 20:22:37 | 452 | 09:20:23 | 200 | 08:00:08

本数据集的代码仓库：该GitHub仓库提供了原始视频下载与剪辑为视频片段的脚本。如有相关疑问，请联系[e-mail]。