qualcomm-interactive-cooking-dataset-counterfactual-mistakes

# Qualcomm Interactive Cooking Dataset: Ego Counterfactual Mistakes ## Description This synthetic dataset contains mistake-intervention annotations for interactive cooking guidance. Each row contains video segment with instruction/feedback text pairs and their timestamps. ## Dataset Details Files: - `annotations.json` Release statistics: - Total rows: 25,087 - Unique videos (dataset + video_id): 1,110 Rows by source dataset: - `CaptainCook4D`: 4,969 - `Ego4D`: 13,847 - `Ego-Exo4D`: 6,271 ## Dataset Collection Process The dataset is generated using Qwen3-VL-32B-Instruct and Gemini-2.5-Pro. ## Data Format Each row in `annotations.json` has schema: ```json { "dataset": "Ego4D", "video_id": "39d087b0-afc2-47d8-ba91-b70dd8fab90e", "start_time": 1191.26881, "end_time": 1204.66377, "mistake_type": "preparation_error", "texts": [ "Microwave the water until boiling.", "For this step, please boil the water in the microwave instead of on the stovetop." ], "text_timestamps": [1191.26881, 1196.26881], "text_types": ["Instruction", "Feedback"] } ``` ## Usage ```python from datasets import load_dataset ds = load_dataset("qualcomm/qualcomm-interactive-cooking-dataset-counterfactual-mistakes", split="train") row = ds[0] source_dataset = row["dataset"] video_id = row["video_id"] texts = row["texts"] text_types = row["text_types"] text_timestamps = row["text_timestamps"] ``` ## Dataset License This dataset is released for research purposes only. Use of the dataset is subject to the license terms of the Qualcomm Interactive Cooking Dataset. Please refer to the [accompanying license documentation](LICENSE.pdf) for full terms, conditions, and usage restrictions. ## Dataset Citation Instructions If you use this dataset, please cite: ```bibtex @inproceedings{interactivecooking, title = {Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?}, author = {Apratim Bhattacharyya and Bicheng Xu and Sanjay Haresh and Reza Pourreza and Litian Liu and Sunny Panchal and Leonid Sigal and Roland Memisevic}, booktitle = {NeurIPS}, year = {2025} } ```