qualcomm-interactive-cooking-dataset-ego-mistake-corrections

# Qualcomm Interactive Cooking Dataset: Ego Mistake Corrections Benchmark ## Description This dataset contains cooking videos with timestamped instruction and feedback for task guidance. Each row corresponds to one video and provides aligned lists of utterance text, utterance type, and timestamp. ## Dataset Details Release files: - `annotations/annotations.json` - `videos/*.MP4` Release statistics: - Total videos: 40 - Total released annotations: 1,597 Text type counts in `annotations_release.json`: - `Instruction`: 562 - `Success`: 560 - `Feedback`: 397 - `Instruction-Clarification`: 26 - `User-Question`: 22 - `User-Comment`: 30 ## Dataset Collection Process Annotations are timestamped and aligned with each video. They are manually annotated. ## Data Format Each row in `annotations.json` has schema: ```json { "video_id": "GX010021", "texts": [ "Let's first preheat the oven to 350 degrees Fahrenheit.", "Great Job!" ], "text_types": [ "Instruction", "Success" ], "text_timestamps": [ 4.418, 15.982 ] } ``` Video asset for a sample `video_id`: - `videos/<video_id>.MP4` ## Usage ```python from datasets import load_dataset ds = load_dataset("qualcomm/qualcomm-interactive-cooking-dataset-ego-mistake-corrections", split="test") row = ds[0] video_id = row["video_id"] texts = row["texts"] text_types = row["text_types"] text_timestamps = row["text_timestamps"] # Video path convention: # f"videos/{video_id}.MP4" ``` ## Dataset License This dataset is released for research purposes only. Use of the dataset is subject to the license terms of the Qualcomm Interactive Cooking Dataset. Please refer to the [accompanying license documentation](LICENSE.pdf) for full terms, conditions, and usage restrictions. ## Dataset Citation Instructions If you use this dataset, please cite: ```bibtex @inproceedings{streamcooking, title = {Streaming Interventions: Can Video LLMs Correct Mistakes as They Occur?}, author = {Apratim Bhattacharyya and Shweta Mahajan and Sanjay Haresh and Risheek Garrepalli and Rajeev Yasarla and Litian Liu and Reza Pourreza and Roland Memisevic}, booktitle = {arXiv}, year = {2026} } ``` ## Qualcomm AI Research At Qualcomm AI Research, we are advancing AI to make its core capabilities – perception, reasoning, and action – ubiquitous across devices. Our mission is to make breakthroughs in fundamental AI research and scale them across industries. By bringing together some of the best minds in the field, we’re pushing the boundaries of what’s possible and shaping the future of AI. Qualcomm AI Research continues to invest in and support deep-learning research in computer vision. The publication of this dataset for use by the AI research community is one of our many initiatives. Find out more about Qualcomm AI Research. For any questions or technical support, please contact us at [e-mail] Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.