Jazzcharles/ego4d_train_pair_howto100m

---
license: apache-2.0
task_categories:
- video-classification
language:
- en
size_categories:
- 1B<n<10B
---

## 📙 Overview

The metadata for Ego4d training set, with paired howto100m video clips. The ego-exo pair is constructed by choosing the ones with shared nouns/verbs.

Each sample represents a short video clip, which consists of
- vid: the initial video id.
- start_second: the start timestamp of the narration.
- end_second: the end timestamp of the narration.
- text: the original narration.
- noun: a list containing the index of nouns in the Ego4d noun vocabulary.
- verb: a list containing the index of verbs in the Ego4d verb vocabulary.
- nv_index: a list of numbers indexing the [paired howto100m clips](https://huggingface.co/datasets/Jazzcharles/HowTo100M_llama3_refined_caption).

The noun and verb vocabularies can be found in EgoVLP (https://drive.google.com/drive/folders/16fUv5rrZmt06Ty3QAEweDpveC-84RI9Z).

## 🏋️ How-To-Use

Please refer to code [EgoInstructor](https://github.com/Jazzcharles/Egoinstructor/) for details.

## 🎓 Citation
```
@article{xu2024retrieval,
title={Retrieval-augmented egocentric video captioning},
author={Xu, Jilan and Huang, Yifei and Hou, Junlin and Chen, Guo and Zhang, Yuejie and Feng, Rui and Xie, Weidi},
journal={arXiv preprint arXiv:2401.00789},
year={2024}
}
```

原始信息汇总

数据集概述

基本信息

• 许可证: Apache-2.0
• 任务类别: 视频分类
• 语言: 英语
• 数据集大小: 1B<n<10B

数据集内容

• 描述: 该数据集为Ego4d训练集，包含与howto100m视频片段配对的数据。每个样本代表一个短视频片段，包含以下信息：
  • vid: 初始视频ID。
  • start_second: 叙述开始的时间戳。
  • end_second: 叙述结束的时间戳。
  • text: 原始叙述文本。
  • noun: 包含Ego4d名词词汇表中名词索引的列表。
  • verb: 包含Ego4d动词词汇表中动词索引的列表。
  • nv_index: 索引配对howto100m片段的数字列表。

词汇表位置

• 名词和动词词汇表: 可在EgoVLP中找到，链接为EgoVLP。