DStardust/EgoCoT-Bench

---
pretty_name: "EgoCoT-Bench"
tags:
- video
- multimodal
- benchmark
- reasoning
task_categories:
- visual-question-answering
language:
- en
license: apache-2.0
size_categories:
- 1K<n<10K
---
# EgoCoT-Bench

EgoCoT-Bench is a benchmark for grounded and verifiable reasoning in egocentric videos.

## Overview

This repository contains the **public release** of EgoCoT-Bench. The release is organized for benchmark evaluation rather than model training.

- Total samples in the processed benchmark: **3172**
- Total unique videos in the processed benchmark: **351**
- Public dev samples: **300**
- Public dev unique videos: **193**
- Public test samples: **2872**
- Public test unique videos: **351**

## Files

- `public_dev.json`: public development split with **question, choices, answer, and CoT**.
- `public_test.json`: public test split with **question and choices only**.
- `media/`: video files referenced by the public splits.
- `LICENSE`: Apache License 2.0 text.

The official hidden test labels are **not included** in this public repository.

## Data Fields

Each sample may include the following fields:

- `qid`: unique sample identifier assigned for release
- `question`: canonical final question text
- `choices`: canonical final multiple-choice options
- `answer`: canonical final answer (public dev only)
- `cot`: canonical final chain-of-thought style rationale annotation (public dev only)
- `media_id`: video identifier
- `source`: relative path to the video file
- `big_category_en`: high-level task group
- `sub_category_en`: fine-grained subtask
- `type`: original task type label

## Access and Licensing

The annotation files, benchmark split files, metadata files, and evaluation-related release files in this repository are released under the **apache-2.0**.

A copy of the license text is provided in the `LICENSE` file in this repository.

The official hidden test labels are **not included** in this public repository.

If media files are included in or accessed through this benchmark release, users are responsible for ensuring that their use of such media complies with the licenses and terms of the original media sources.

- Homepage: https://dstardust.github.io/EgoCoT/
- Paper: EgoCoT-Bench: Benchmarking Grounded and Verifiable Operation-Centric Chain of Thought Reasoning for MLLMs

## Loading Example

```python
import json

with open("public_dev.json", "r", encoding="utf-8") as f:
public_dev = json.load(f)

with open("public_test.json", "r", encoding="utf-8") as f:
public_test = json.load(f)

print(len(public_dev), len(public_test))
print(public_dev[0].keys())
```