A Dataset for Earth Observation Question–Answering Using a RAG-Based Model

A Dataset for Earth Observation Question–Answering Using a RAG-Based Model

Overview

This dataset is created as part of the paper:

Knowledge Graph-Enhanced Retrieval-Augmented Generation for Earth Observation accepted at the DARES25 Workshop (2025)Roxanne El Baff, Ben Schluckebier and Tobias Hecking

It comprises two complementary CSV files:

DARES25_EarthObservation_generated_questions_v1.0.csv — Contains 70 domain-specific questions generated using GPT-4o in a zero-shot manner. Each question combines a topic randomly sampled from 14 NASA GCMD taxonomy areas (e.g., Atmosphere, Cryosphere, Oceans) with one of five intent categories (Exploratory, Comparative, Descriptive, Causal, Relational) to ensure diversity. 

DARES25_EarthObservation_QA_RAG_results_v1.0.csv — Contains the answers to each question (from 1.) produced by our Retrieval-Augmented Generation (RAG) pipeline. For each question, this file includes retrieval parameters, retrieved documents, structured context, and multiple answer types (RAG, zero-shot, and two-step refined RAG), along with metadata such as model, temperature, and timestamp.

# hello

I Question Generation - 📄DARES25_EarthObservation_generated_questions_v1.0.csv

a. File Structure

The file contains 70 rows, each has the generated question and the corresponding metadata:

idx — A unique ID for the question.

intent — Intent category (Exploratory, Comparative, Descriptive, Causal, Relational).

topic — NASA GCMD topic the question is based on.

term — NASA GCMD term the question is based on.

context — The formatted context created from intent, topic, and term. This context is included in the instruction prompt to generate a question.

instructions — The LLM prompt used to generate the question (includes the prompt displayed above and the corresponding `context`).

question — Generated question text.

b. Prompts

The 70 questions are generated using GPT-4o in a zero-shot manner. Below we show the prompt used:

System PromptYou are an expert scientist and a critical thinker.
Your task is to generate a realistic, concise scientific question with given criteria.

User Prompt

### Question Criteria:

INTENT: {intent_category} — {intent_description}

TOPIC:

The topics are extracted from the NASA GCMD Taxonomy. The TOPIC is described and few of its SUBJECT AREAS.

<topic>
{context}
</topic>

### Instructions

The question must strictly reflect the stated INTENT and be centered on the specified TOPIC along with its SUBJECT AREAS. Do not include any background or explanation—just the question.
This question will be used to evaluate the performance of a scientific QA system under Retrieval-Augmented Generation (RAG) and zero-shot prompting conditions.

### Question:

II LLM Answers -📄DARES25_EarthObservation_QA_RAG_results_v1.0.csv

a. File Structure

The file has 140 rows, where each row contains all the answers (Zero-Shot, RAG and 2-steps-RAG) for each model (i.e., Mistral Small or Llama 70B):

idx — Row index per question-answers-model

model — Model used to generate the answer (e.g., mistral-small-2503).

temperature — Generation temperature used for the model. The -1 values means that the temperature was not set and the default values was used, 0.7.

question — Question posed to the system.

aql_params — Parameters for the retrieval step.

query — Search query sent to the index.

aql_results — Retrieved documents (titles, snippets, etc.).

context — Extracted context passages used in RAG.

rag_answer — Single-step RAG-generated answer.

zero_shot_answer — Zero-shot model answer without retrieval.

structured_context — Markdown-like formatted related documents with metadata.

rag_two_steps_answer — Refined answer after a two-step RAG process.

timestamp — Timestamp when the QA run occurred.

b. Prompts

We used three related prompt setups to generate the answers:

- System Prompt (Same across all 3 approaches: zero-shot, Rag and 2-steps-rag)

System Prompt

You are an expert assistant specialized in Earth Observation (EO) science.Your task is to answer scientific questions with a language suitable for research contexts. 

- Prompt fot Zero-Shot, generates `zero_shot_answer`

User Prompt

### Question:{question}

### Answer:

- Prompt for RAG-Based model that uses `structured_context`, and generates `rag_answer`

User Prompt

### QUESTION

{question}

### CONTEXT

The context contains publications and their related topics extracted from an EO Knowledge Graph

{context}

### INSTRUCTIONS

Read instructions first then answer:- Use context *only when relevant* "(from context)".- Use your knowledge to add missing EO knowledge "(from knowledge)".

### Answer

- Prompt for (2RAG) refining answer from the Zero-Shot with `structured_context`, generates `rag_two_steps_answer`

You answered a question with the following draft answer, using your knowledge.Now refine this answer using the provided context.

### YOUR DRAFT ANSWER

-  Question: {question}

- Draft Answer:

{draft_answer}

### CONTEXT

The context contains publications and their related topics extracted from an EO Knowledge Graph

{context}

### INSTRUCTIONS

- Keep correct details from your draft.- Add missing relevant details from context.- If context contradicts your draft, revise your answer.

Read the instructions then answer

### Refine Answer

Code and new updates are available on GitHub.