soynade-research/kallaama-retrival-eval-corpus

---
dataset_info:
features:
- name: _id
dtype: string
- name: text
dtype: string
splits:
- name: train
num_bytes: 434162.76666666666
num_examples: 445
download_size: 264922
dataset_size: 434162.76666666666
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

### Fleurs-Retrieval-Eval

**Fleurs-Retrieval-Eval** is a cross-lingual speech-to-text retrieval evaluation dataset designed to assess French document retrieval from Wolof speech queries. It is derived from the test split of SIB-Fleurs, a multilingual spoken language understanding benchmark based on the FLEURS corpus.

The dataset is constructed using fully natural speech data to provide a realistic evaluation setting, in contrast to synthetic training corpora. For each Wolof speech sample, its French translation is used as a query to retrieve relevant French documents from the web. The retrieved documents are then manually filtered to remove irrelevant or low-quality results. The retained documents are kept in their original form, without cleaning or shortening, in order to reflect real-world information retrieval conditions.

The resulting dataset contains 166 pairs of Wolof speech queries and corresponding French documents.

Fleurs-Retrieval-Eval is intended exclusively for evaluation and benchmarking. It enables the assessment of cross-lingual and cross-modal retrieval systems in low-resource settings, particularly for speech-based access to written information.

The dataset is primarily used to evaluate retrieval quality using ranking metrics such as nDCG@k and serves as a reference benchmark for speech-to-document retrieval between Wolof and French.

### Fleurs-Retrieval-Eval-Corpus
This repository contains the raw documents to retrieve. The associated queries are here: [soynade-research/kallaama-retrival-eval-queries](https://huggingface.co/datasets/soynade-research/kallaama-retrival-eval-queries). The mapping between queries and documents is in this repository: [soynade-research/kallaama-retrival-eval-qrels](https://huggingface.co/datasets/soynade-research/kallaama-retrival-eval-qrels/)