Dora WalkingTours Dataset (ICLR 2024)

Self-supervised learning has unlocked the potential of scaling up pretraining to billions of images, since annotation is unnecessary. But are we making the best use of data? How more economical can we be? In this work, we attempt to answer this question by making two contributions. First, we investigate first-person videos and introduce a "Walking Tours" dataset. These videos are high-resolution, hours-long, captured in a single uninterrupted take, depicting a large number of objects and actions with natural scene transitions. They are unlabeled and uncurated, thus realistic for self-supervision and comparable with human learning.Second, we introduce a novel self-supervised image pretraining method tailored for learning from continuous videos. Reference:Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled video. Shashanka Venkataramanan, Mamshad Nayeem Rizve, João Carreira, Yuki M. Asano, Yannis Avrithis. In: International Conference on Learning Representations 2024

自监督学习（Self-supervised Learning）无需标注数据，从而解锁了将预训练规模拓展至数十亿张图像的潜力。但我们是否已实现数据的最优利用？我们还能否进一步提升数据使用的经济性？本研究旨在解答上述问题，主要贡献有二：其一，我们对第一视角视频展开研究，并构建了「行走之旅（Walking Tours）」数据集。该数据集包含的视频均为高分辨率、长时长的单镜头无间断拍摄内容，展现了大量物体与动作，且带有自然的场景过渡效果。这些视频未标注且未经过人工筛选，因此适配真实的自监督学习场景，可与人类学习过程相媲美。其二，我们提出了一种专为连续视频学习设计的新型自监督图像预训练方法。参考文献：《ImageNet是否值得一段视频？从一段长未标注视频中学习高性能图像编码器》，作者为Shashanka Venkataramanan、Mamshad Nayeem Rizve、João Carreira、Yuki M. Asano、Yannis Avrithis，发表于国际学习表征会议（International Conference on Learning Representations）2024

搜集汇总

背景与挑战

背景概述

Dora WalkingTours Dataset是一个包含多个高分辨率、长时间第一人称步行游览视频的数据集，适用于自监督学习研究。数据集覆盖多个城市和野生动物场景，总大小为81.23 GB，包含10个视频文件。