NavQA_Revised

# NavQA Revised NavQA Revised is a re-annotated version of the NaVQA dataset released with [NVIDIA ReMEmbR](https://github.com/NVIDIA-AI-IOT/remembr). The original annotations are distributed in [`remembr/data/navqa/data.csv`](https://github.com/NVIDIA-AI-IOT/remembr/blob/main/remembr/data/navqa/data.csv). This repository provides the revised annotations as JSONL files: ```text navqa.jsonl navqa_over_sequence.jsonl ``` Each line is one question-answer example. `navqa.jsonl` is the primary revised annotation file. `navqa_over_sequence.jsonl` uses the same schema and examples, but uses the beginning of the full sequence as the query start point. In this variant, the query does not need to consider selecting a shorter time segment from within the sequence. ## Dataset Details | Item | Value | | --- | --- | | Dataset type | Re-annotated navigation video question answering annotations | | Language | English | | Task categories | Question answering, video question answering, embodied navigation reasoning | | License | Apache 2.0 | | Total files | 2 JSONL files | | Split | `test` only | | Examples per file | 210 | | Total rows across both configs | 420 | | Source dataset | NaVQA from ReMEmbR | | Source sequences | CODa sequence ids `0`, `3`, `4`, `6`, `16`, `21`, `22` | | Examples per sequence | 30 | ## Configurations | Config name | Data file | Split | Examples | Query start point | | --- | --- | --- | ---: | --- | | `navqa` | `navqa.jsonl` | `test` | 210 | Start of the selected QA time window | | `navqa_over_sequence` | `navqa_over_sequence.jsonl` | `test` | 210 | Start of the full sequence | ## What Changed Compared with the original NaVQA annotations, this version makes two main corrections. 1. **Timezone correction** The original timestamps are interpreted using the local timezone of the UT Austin campus where the data was collected (`America/Chicago`). We convert Unix timestamps to the correct local time before matching them with navigation events and before inserting them into the natural-language query. 2. **2 fps matching and direct query generation** Instead of using a coarser caption interval, examples are matched at 2 fps. The final query text is generated directly from the corrected temporal and spatial context, including the start time, current time, and current robot pose when available. ## Dataset Summary Each file contains 210 examples: | Split by video length | Count | | --- | ---: | | `SHORT` | 70 | | `MEDIUM` | 70 | | `LONG` | 70 | Answer types: | Type | Count | | --- | ---: | | `position` | 71 | | `binary` | 67 | | `text` | 33 | | `time` | 30 | | `duration` | 9 | ## File Variants - `navqa.jsonl`: The query start time is the start of the selected QA time window. - `navqa_over_sequence.jsonl`: The query start time is the beginning of the corresponding sequence. This version treats the query as being asked over the whole sequence history, so there is no need to choose an additional temporal segment inside the sequence. ## Data Format Each JSON object has the following fields: | Field | Description | | --- | --- | | `id` | Unique example identifier. | | `seq_id` | CODa sequence id associated with the example. | | `length_category` | Video-length bucket: `SHORT`, `MEDIUM`, or `LONG`. | | `length` | Duration of the query window in seconds. | | `start_time` | Unix timestamp for the start of the query window. | | `end_time` | Unix timestamp for the end/current time of the query window. | | `file_info` | Source frame/window filenames used for QA and context matching. | | `question` | Final natural-language query generated from the corrected context. | | `type` | Expected answer type: `position`, `binary`, `text`, `time`, or `duration`. | | `answers` | Structured ground-truth answer. | | `question_category` | Original or revised semantic category when available. | | `answer_timestamp` | Timestamp(s) of the evidence used to answer the question. | | `text_answer` | Optional free-form answer text. | | `parsable_answer` | Normalized answer pattern for evaluation. | Example: ```json { "id": "LONG_5a6cef0c-175d-4779-bfd4-f7b193fb9039", "seq_id": 0, "length_category": "LONG", "question": "You started moving at 2023-01-16 09:55:32. The current time is 2023-01-16 10:03:02 and you are located at [1.42, 2.28, -0.17].\nWhen did you leave the building?", "type": "time", "answers": { "text": ["4.07 minutes ago"], "time": 4.07 }, "answer_timestamp": "9:59:00", "parsable_answer": "[minutes] ago" } ``` ## Loading the Data ```python import json examples = [] with open("navqa.jsonl", "r", encoding="utf-8") as f: for line in f: examples.append(json.loads(line)) print(len(examples)) print(examples[0]["question"]) ``` ## Intended Use This dataset is intended for evaluating navigation-oriented video question answering systems and embodied agents that reason over long-horizon spatial and temporal memory. It is especially useful for models that need to answer: - spatial questions that return a goal position, - temporal questions that return a time or elapsed duration, - descriptive questions that return text, - binary questions about whether an event or object appears in the navigation history. ## Relationship to ReMEmbR and CODa ReMEmbR introduces NaVQA as an evaluation dataset for long-horizon spatio-temporal robot memory. The upstream evaluation instructions describe NaVQA as 210 examples built from selected CODa sequences, with 30 questions per sequence. This repository only provides the revised QA annotations. It does not include the original CODa sensor data, images, captions, or ReMEmbR model outputs. To reconstruct the full evaluation pipeline, refer to the upstream [ReMEmbR repository](https://github.com/NVIDIA-AI-IOT/remembr) and its [evaluation instructions](https://github.com/NVIDIA-AI-IOT/remembr/blob/main/eval.md). ## License This repository is released under the Apache 2.0 license. The data is derived from the NaVQA annotations.