TimelyEventsBenchmark/TiEBe

---
language:
- en
dataset_info:
features:
- name: year
dtype: string
- name: month
dtype: string
- name: event_desc
dtype: string
- name: text
dtype: string
- name: question
dtype: string
- name: answer
dtype: string
splits:
- name: world
num_bytes: 43850273
num_examples: 4472
- name: eua
num_bytes: 26057335
num_examples: 2945
- name: brazil
num_bytes: 19187973
num_examples: 1680
- name: france
num_bytes: 5180668
num_examples: 634
- name: portugal
num_bytes: 4660386
num_examples: 656
- name: china
num_bytes: 5018316
num_examples: 849
download_size: 28467887
dataset_size: 103954951
configs:
- config_name: default
data_files:
- split: world
path: data/world-*
- split: eua
path: data/eua-*
- split: brazil
path: data/brazil-*
- split: france
path: data/france-*
- split: portugal
path: data/portugal-*
- split: china
path: data/china-*
---

# Dataset Card for TiEBe

## Dataset Summary

**TiEBe (Timely Events Benchmark)** is a dataset designed to assess the factual recall and regional knowledge representation of large language models (LLMs) concerning significant global and regional events. It contains over 11,000 question-answer pairs derived from retrospective Wikipedia pages and their references, spanning six geographic regions (World, USA, Brazil, China, Portugal, and France) and covering the years 2015–2024. TiEBe supports the evaluation of LLMs in two critical areas: geographical disparities in knowledge and continual learning of time-sensitive information.

## Supported Tasks

- **Task:** Factual Question Answering
- Input: Question related to a specific event.
- Output: Model-generated answer.

## Languages

Questions and answers are provided in **English** to ensure consistency and comparability across regions and models, even though the source documents may be in other languages.

## Dataset Structure

### Data Fields

- **Region:** The geographic focus of the event (e.g., World, USA, Brazil, etc.).
- **Year:** The year of the event (as extracted from Wikipedia).
- **Event Description:** A brief summary of the event (as extracted from Wikipedia).
- **Question:** A question designed to assess factual recall of the event.
- **Answer:** The correct answer to the question.

### Data Splits

The dataset does not include predefined splits. Users can define their own train/test/validation splits as required.

## Dataset Creation

### Source Data

- **Event Sources:** Wikipedia retrospective pages (2015–2024) for five countries and global events.
- **References:** Citation links from Wikipedia pages, typically journalistic sources.

### Data Generation

1. **Event Extraction:** Extracted notable events from retrospective pages and references.
2. **QA Generation:** Generated question-answer pairs using GPT-4 with a standardized prompt to ensure relevance and clarity.

## Key Features

- Covers a wide temporal range (2015–2024) for studying the evolution of model knowledge.
- Includes events from multiple regions, highlighting geographical disparities.

## Usage

### Example

```python
from datasets import load_dataset

# Load the TiEBe dataset
dataset = load_dataset("TimelyEventsBenchmark/TiEBe", split="world")

print(dataset[0])
# {
# "year": "2019",
# "month": "02",
# "event_desc": "U.S. President Donald Trump confirms that the U.S. will leave the Intermediate-Range Nuclear....",
# "text": "INF nuclear treaty: Russia follows US in suspending pact...",
# "question": "What action did the United States take regarding the...",
# "answer": "In February 2019, the United States formally announced...",
# }

```

## Citation

paper coming soon