claudefitz/WaxalNLP

---
language_creators:
- creator_1
language:
- ach
- aka
- amh
# - bam
- bau
- dag
- dga
- ewe
- fat
# - fuf
- ful
- hau
- ibo
- kik
- kpo
- lin
- lug
- luo
- mas
- mlg
- nyn
- orm
- pcm
- sid
- sna
- sog
- swa
- tir
- twi
- wal
# - wol
- yor
license:
- cc-by-sa-4.0
- cc-by-4.0
multilinguality:
- multilingual
source_datasets:
- UGSpeechData
- DigitalUmuganda/AfriVoice
- original
task_categories:
- automatic-speech-recognition
- text-to-speech
pretty_name: Waxal NLP Datasets
arxiv: 2602.02734
annotation_creators:
- human-annotated
- crowdsourced
tags:
- audio
- automatic-speech-recognition
- text-to-speech
configs:
- config_name: ach_asr
data_files:
- split: train
path: data/ASR/ach/ach-train-*
- split: validation
path: data/ASR/ach/ach-validation-*
- split: test
path: data/ASR/ach/ach-test-*
- split: unlabeled
path: data/ASR/ach/ach-unlabeled-*
- config_name: ach_tts
data_files:
- split: train
path: data/TTS/ach/ach-train-*
- split: validation
path: data/TTS/ach/ach-validation-*
- split: test
path: data/TTS/ach/ach-test-*
- config_name: aka_asr
data_files:
- split: train
path: data/ASR/aka/aka-train-*
- split: validation
path: data/ASR/aka/aka-validation-*
- split: test
path: data/ASR/aka/aka-test-*
- split: unlabeled
path: data/ASR/aka/aka-unlabeled-*
- config_name: amh_asr
data_files:
- split: train
path: data/ASR/amh/amh-train-*
- split: validation
path: data/ASR/amh/amh-validation-*
- split: test
path: data/ASR/amh/amh-test-*
- split: unlabeled
path: data/ASR/amh/amh-unlabeled-*
# - config_name: bam_tts
# data_files:
# - split: train
# path: data/TTS/bam/bam-train-*
# - split: validation
# path: data/TTS/bam/bam-validation-*
# - split: test
# path: data/TTS/bam/bam-test-*
- config_name: bau_tts
data_files:
- split: train
path: data/TTS/bau/bau-train-*
- split: validation
path: data/TTS/bau/bau-validation-*
- split: test
path: data/TTS/bau/bau-test-*
- config_name: dag_asr
data_files:
- split: train
path: data/ASR/dag/dag-train-*
- split: validation
path: data/ASR/dag/dag-validation-*
- split: test
path: data/ASR/dag/dag-test-*
- split: unlabeled
path: data/ASR/dag/dag-unlabeled-*
- config_name: dga_asr
data_files:
- split: train
path: data/ASR/dga/dga-train-*
- split: validation
path: data/ASR/dga/dga-validation-*
- split: test
path: data/ASR/dga/dga-test-*
- split: unlabeled
path: data/ASR/dga/dga-unlabeled-*
- config_name: ewe_asr
data_files:
- split: train
path: data/ASR/ewe/ewe-train-*
- split: validation
path: data/ASR/ewe/ewe-validation-*
- split: test
path: data/ASR/ewe/ewe-test-*
- split: unlabeled
path: data/ASR/ewe/ewe-unlabeled-*
- config_name: ewe_tts
data_files:
- split: train
path: data/TTS/ewe/ewe-train-*
- split: validation
path: data/TTS/ewe/ewe-validation-*
- split: test
path: data/TTS/ewe/ewe-test-*
- config_name: fat_tts
data_files:
- split: train
path: data/TTS/fat/fat-train-*
- split: validation
path: data/TTS/fat/fat-validation-*
- split: test
path: data/TTS/fat/fat-test-*
- config_name: ful_asr
data_files:
- split: train
path: data/ASR/ful/ful-train-*
- split: validation
path: data/ASR/ful/ful-validation-*
- split: test
path: data/ASR/ful/ful-test-*
- split: unlabeled
path: data/ASR/ful/ful-unlabeled-*
# - config_name: fuf_tts
# data_files:
# - split: train
# path: data/TTS/fuf/fuf-train-*
# - split: validation
# path: data/TTS/fuf/fuf-validation-*
# - split: test
# path: data/TTS/fuf/fuf-test-*
- config_name: ful_tts
data_files:
- split: train
path: data/TTS/ful/ful-train-*
- split: validation
path: data/TTS/ful/ful-validation-*
- split: test
path: data/TTS/ful/ful-test-*
- config_name: hau_tts
data_files:
- split: train
path: data/TTS/hau/hau-train-*
- split: validation
path: data/TTS/hau/hau-validation-*
- split: test
path: data/TTS/hau/hau-test-*
- config_name: ibo_tts
data_files:
- split: train
path: data/TTS/ibo/ibo-train-*
- split: validation
path: data/TTS/ibo/ibo-validation-*
- split: test
path: data/TTS/ibo/ibo-test-*
- config_name: kik_tts
data_files:
- split: train
path: data/TTS/kik/kik-train-*
- split: validation
path: data/TTS/kik/kik-validation-*
- split: test
path: data/TTS/kik/kik-test-*
- config_name: kpo_asr
data_files:
- split: train
path: data/ASR/kpo/kpo-train-*
- split: validation
path: data/ASR/kpo/kpo-validation-*
- split: test
path: data/ASR/kpo/kpo-test-*
- split: unlabeled
path: data/ASR/kpo/kpo-unlabeled-*
- config_name: lin_asr
data_files:
- split: train
path: data/ASR/lin/lin-train-*
- split: validation
path: data/ASR/lin/lin-validation-*
- split: test
path: data/ASR/lin/lin-test-*
- split: unlabeled
path: data/ASR/lin/lin-unlabeled-*
- config_name: lug_asr
data_files:
- split: train
path: data/ASR/lug/lug-train-*
- split: validation
path: data/ASR/lug/lug-validation-*
- split: test
path: data/ASR/lug/lug-test-*
- split: unlabeled
path: data/ASR/lug/lug-unlabeled-*
- config_name: lug_tts
data_files:
- split: train
path: data/TTS/lug/lug-train-*
- split: validation
path: data/TTS/lug/lug-validation-*
- split: test
path: data/TTS/lug/lug-test-*
- config_name: luo_tts
data_files:
- split: train
path: data/TTS/luo/luo-train-*
- split: validation
path: data/TTS/luo/luo-validation-*
- split: test
path: data/TTS/luo/luo-test-*
- config_name: mas_asr
data_files:
- split: train
path: data/ASR/mas/mas-train-*
- split: validation
path: data/ASR/mas/mas-validation-*
- split: test
path: data/ASR/mas/mas-test-*
- split: unlabeled
path: data/ASR/mas/mas-unlabeled-*
- config_name: mlg_asr
data_files:
- split: train
path: data/ASR/mlg/mlg-train-*
- split: validation
path: data/ASR/mlg/mlg-validation-*
- split: test
path: data/ASR/mlg/mlg-test-*
- split: unlabeled
path: data/ASR/mlg/mlg-unlabeled-*
- config_name: nyn_asr
data_files:
- split: train
path: data/ASR/nyn/nyn-train-*
- split: validation
path: data/ASR/nyn/nyn-validation-*
- split: test
path: data/ASR/nyn/nyn-test-*
- split: unlabeled
path: data/ASR/nyn/nyn-unlabeled-*
- config_name: nyn_tts
data_files:
- split: train
path: data/TTS/nyn/nyn-train-*
- split: validation
path: data/TTS/nyn/nyn-validation-*
- split: test
path: data/TTS/nyn/nyn-test-*
- config_name: orm_asr
data_files:
- split: train
path: data/ASR/orm/orm-train-*
- split: validation
path: data/ASR/orm/orm-validation-*
- split: test
path: data/ASR/orm/orm-test-*
- split: unlabeled
path: data/ASR/orm/orm-unlabeled-*
- config_name: pcm_tts
data_files:
- split: train
path: data/TTS/pcm/pcm-train-*
- split: validation
path: data/TTS/pcm/pcm-validation-*
- split: test
path: data/TTS/pcm/pcm-test-*
- config_name: sid_asr
data_files:
- split: train
path: data/ASR/sid/sid-train-*
- split: validation
path: data/ASR/sid/sid-validation-*
- split: test
path: data/ASR/sid/sid-test-*
- split: unlabeled
path: data/ASR/sid/sid-unlabeled-*
- config_name: sna_asr
data_files:
- split: train
path: data/ASR/sna/sna-train-*
- split: validation
path: data/ASR/sna/sna-validation-*
- split: test
path: data/ASR/sna/sna-test-*
- split: unlabeled
path: data/ASR/sna/sna-unlabeled-*
- config_name: tir_asr
data_files:
- split: train
path: data/ASR/tir/tir-train-*
- split: validation
path: data/ASR/tir/tir-validation-*
- split: test
path: data/ASR/tir/tir-test-*
- split: unlabeled
path: data/ASR/tir/tir-unlabeled-*
- config_name: sog_asr
data_files:
- split: train
path: data/ASR/sog/sog-train-*
- split: validation
path: data/ASR/sog/sog-validation-*
- split: test
path: data/ASR/sog/sog-test-*
- split: unlabeled
path: data/ASR/sog/sog-unlabeled-*
- config_name: swa_tts
data_files:
- split: train
path: data/TTS/swa/swa-train-*
- split: validation
path: data/TTS/swa/swa-validation-*
- split: test
path: data/TTS/swa/swa-test-*
- config_name: twi_tts
data_files:
- split: train
path: data/TTS/twi/twi-train-*
- split: validation
path: data/TTS/twi/twi-validation-*
- split: test
path: data/TTS/twi/twi-test-*
- config_name: yor_tts
data_files:
- split: train
path: data/TTS/yor/yor-train-*
- split: validation
path: data/TTS/yor/yor-validation-*
- split: test
path: data/TTS/yor/yor-test-*
- config_name: wal_asr
data_files:
- split: train
path: data/ASR/wal/wal-train-*
- split: validation
path: data/ASR/wal/wal-validation-*
- split: test
path: data/ASR/wal/wal-test-*
- split: unlabeled
path: data/ASR/wal/wal-unlabeled-*
# - config_name: wol_tts
# data_files:
# - split: train
# path: data/TTS/wol/wol-train-*
# - split: validation
# path: data/TTS/wol/wol-validation-*
# - split: test
# path: data/TTS/wol/wol-test-*
dataset_info:
- config_name: ach_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ach_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: aka_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: amh_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
# - config_name: bam_tts
# features:
# - name: id
# dtype: string
# - name: speaker_id
# dtype: string
# - name: text
# dtype: string
# - name: locale
# dtype: string
# - name: gender
# dtype: string
# - name: audio
# dtype: audio
- config_name: bau_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: dag_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: dga_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ewe_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ewe_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: fat_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ful_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: fuf_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ful_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: hau_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ibo_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: kik_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: kpo_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lin_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lug_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lug_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: luo_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: mas_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: mlg_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: nyn_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: nyn_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: orm_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: pcm_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: sid_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: sna_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: sog_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: swa_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: tir_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: twi_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: wal_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
# - config_name: wol_tts
# features:
# - name: id
# dtype: string
# - name: speaker_id
# dtype: string
# - name: text
# dtype: string
# - name: locale
# dtype: string
# - name: gender
# dtype: string
# - name: audio
# dtype: audio
- config_name: yor_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
---

# Waxal Datasets

The WAXAL dataset is a large-scale multilingual speech corpus for African languages, introduced in the paper [WAXAL: A Large-Scale Multilingual African Language Speech Corpus](https://huggingface.co/papers/2602.02734).

## Table of Contents

- [Dataset Description](#dataset-description)
- [ASR Dataset](#asr-dataset)
- [TTS Dataset](#tts-dataset)
- [How to Use](#how-to-use)
- [Dataset Structure](#dataset-structure)
- [ASR Data Fields](#asr-data-fields)
- [TTS Data Fields](#tts-data-fields)
- [Data Splits](#data-splits)
- [Dataset Curation](#dataset-curation)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Additional Information](#additional-information)
- [Citation](#citation)

## Dataset Description

The Waxal project provides datasets for both Automated Speech Recognition (ASR)
and Text-to-Speech (TTS) for African languages. The goal of this dataset's
creation and release is to facilitate research that improves the accuracy and
fluency of speech and language technology for these underserved languages, and
to serve as a repository for digital preservation.

The Waxal datasets are collections acquired through partnerships with Makerere
University, The University of Ghana, Digital Umuganda, Media Trust, Loud and
Clear, and AIMS Senegal. Acquisition was funded by Google and the Gates
Foundation under an agreement to make the dataset openly accessible. The Senegalese
languages (Wolof and Pular) were provided by AIMS Senegal.

### ASR Dataset

The Waxal ASR dataset is a collection of data in 19 African languages. It
consists of approximately 1,250 hours of transcribed natural speech from a wide
variety of voices. The 19 languages in this dataset represent over 100 million
speakers across 40 Sub-Saharan African countries.

Provider | Languages | License
:------------------ | :--------------------------------------- | :------------:
Makerere University | Acholi, Luganda, Masaaba, Nyankole, Soga | `CC-BY-SA-4.0`
University of Ghana | Akan, Ewe, Dagbani, Dagaare, Ikposo | `CC-BY-4.0`
Digital Umuganda | Fula, Lingala, Shona, Malagasy, Amharic, Oromo, Sidama, Tigrinya, Wolaytta | `CC-BY-SA-4.0`

### TTS Dataset

The Waxal TTS dataset is a collection of text-to-speech data in 17 African
languages. It consists of over 180 hours of high-quality, single-speaker recordings reading phonetically balanced scripts.

Provider | Languages | License
:------------------ | :----------------------------------- | :------------:
Makerere University | Acholi, Luganda, Kiswahili, Nyankole | `CC-BY-SA-4.0`
University of Ghana | Akan (Fante, Twi), Baoule, Ewe | `CC-BY-4.0`
Media Trust | Fula, Igbo, Hausa, Yoruba, Nigerian Pidgin | `CC-BY-SA-4.0`
Loud and Clear | Kikuyu, Luganda, Luo, Swahili | `CC-BY-SA-4.0`
AIMS Senegal | Bambara, Pular, Wolof | `CC-BY-SA-4.0`

### How to Use

The `datasets` library allows you to load and pre-process your dataset in pure
Python, at scale.

First, ensure you have the necessary dependencies installed to handle audio
data. You will need `ffmpeg` installed on your system.

**Google Colab / Ubuntu**
```bash
sudo apt-get install ffmpeg
pip install datasets[audio]
```

**macOS**
```bash
brew install ffmpeg
pip install datasets[audio]
```

**Windows**
Download and install from [ffmpeg.org](https://ffmpeg.org/download.html) and ensure it's in your PATH.
```bash
pip install datasets[audio]
```

If you encounter `RuntimeError: Could not load libtorchcodec`, please ensure `ffmpeg` is correctly installed or check for compatibility between your `torch`, `torchaudio`, and `torchcodec` versions.

**Loading ASR Data**

To load ASR data for a specific language, specify the configuration name, e.g.
`sna_asr` for Shona ASR data.

```python
from datasets import load_dataset, Audio

# Load Shona (sna) ASR dataset
asr_data = load_dataset("google/WaxalNLP", "sna_asr")

# Access splits
train = asr_data['train']
val = asr_data['validation']
test = asr_data['test']

# Example: Accessing audio bytes and other fields
example = train[0]
print(f"Transcription: {example['transcription']}")
print(f"Sampling Rate: {example['audio']['sampling_rate']}")
# 'array' contains the decoded audio bytes as a numpy array
print(f"Audio Array Shape: {example['audio']['array'].shape}")
```

**Loading TTS Data**

To load TTS data for a specific language, specify the configuration name, e.g.
`swa_tts` for Swahili TTS data.

```python
from datasets import load_dataset

# Load Swahili (swa) TTS dataset
tts_data = load_dataset("google/WaxalNLP", "swa_tts")

# Access splits
train = tts_data['train']
```

## Dataset Structure

### ASR Data Fields

```python
{
'id': 'sna_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'transcription': '...',
'language': 'sna',
'gender': 'Female',
}
```

* **id**: Unique identifier.
* **speaker_id**: Unique identifier for the speaker.
* **audio**: Audio data.
* **transcription**: Transcription of the audio.
* **language**: ISO 639-2 language code.
* **gender**: Speaker gender ('Male', 'Female', or empty).

### TTS Data Fields

```python
{
'id': 'swa_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'text': '...',
'locale': 'swa',
'gender': 'Female',
}
```

* **id**: Unique identifier.
* **speaker_id**: Unique identifier for the speaker.
* **audio**: Audio data.
* **text**: Text script.
* **locale**: ISO 639-2 language code.
* **gender**: Speaker gender.

### Data Splits

For the **ASR Dataset**, the data with transcriptions is split as follows: *
**train**: 80% of labeled data. * **validation**: 10% of labeled data. *
**test**: 10% of labeled data.

The **unlabeled** split contains all samples that do not have a corresponding
transcription.

The **TTS Dataset** follows a similar structure, with data split into `train`,
`validation`, and `test` sets.

## Dataset Curation

The data was gathered by multiple partners:

Provider | Dataset | License
:------------------ | :------------------------------------------------------- | :------
University of Ghana | [UGSpeechData](https://doi.org/10.57760/sciencedb.22298) | `CC BY 4.0`
Digital Umuganda | [AfriVoice](DigitalUmuganda/AfriVoice) | `CC-BY-SA 4.0`
Makerere University | [Yogera Dataset](https://doi.org/10.7910/DVN/BEROE0) | `CC-BY-SA 4.0`
Media Trust | | `CC-BY-SA 4.0`
Loud and Clear | | `CC-BY-SA 4.0`
AIMS Senegal | | `CC-BY-SA 4.0`

## Considerations for Using the Data

Please check the license for the specific languages you are using, as they may
differ between providers.

**Affiliation:** Google Research

## Version and Maintenance

- **Current Version:** 2.0.0
- **Last Updated:** 03/2026

## Citation

```bibtex
@article{waxal2026,
title={WAXAL: A Large-Scale Multilingual African Language Speech Corpus},
author={Anonymous},
journal={arXiv preprint arXiv:2602.02734},
year={2026}
}
```

---
语言创建者:
- creator_1
语言:
- 阿乔利语（ach）
- 阿坎语（aka）
- 阿姆哈拉语（amh）
# - 班巴拉语（bam）
- 鲍勒语（bau）
- 达格班尼语（dag）
- 达加雷语（dga）
- 埃维语（ewe）
- 法塔语（fat）
# - 富富语（fuf）
- 富拉语（ful）
- 豪萨语（hau）
- 伊博语（ibo）
- 基库尤语（kik）
- 伊克波索语（kpo）
- 林加拉语（lin）
- 卢干达语（lug）
- 卢奥语（luo）
- 马萨巴语（mas）
- 马达加斯加语（mlg）
- 恩扬科勒语（nyn）
- 奥罗莫语（orm）
- 尼日利亚皮钦语（pcm）
- 西达马语（sid）
- 绍纳语（sna）
- 索加语（sog）
- 斯瓦西里语（swa）
- 提格雷尼亚语（tir）
- 特威语（twi）
- 沃莱塔语（wal）
# - 沃洛夫语（wol）
- 约鲁巴语（yor）
许可证:
- 知识共享署名-相同方式共享4.0协议（cc-by-sa-4.0）
- 知识共享署名4.0协议（cc-by-4.0）
多语言属性:
- 多语言
源数据集:
- UGSpeechData
- DigitalUmuganda/AfriVoice
- 原始数据集
任务类别:
- 自动语音识别（automatic-speech-recognition）
- 文本转语音（text-to-speech）
友好名称: Waxal NLP 数据集
arXiv编号: 2602.02734
标注创建者:
- 人工标注（human-annotated）
- 众包标注（crowdsourced）
标签:
- 音频（audio）
- 自动语音识别（automatic-speech-recognition）
- 文本转语音（text-to-speech）
配置项:
- 配置名称: ach_asr（阿乔利语自动语音识别）
数据文件:
- 划分集: 训练集（train）
路径: data/ASR/ach/ach-train-*
- 划分集: 验证集（validation）
路径: data/ASR/ach/ach-validation-*
- 划分集: 测试集（test）
路径: data/ASR/ach/ach-test-*
- 划分集: 无标注集（unlabeled）
路径: data/ASR/ach/ach-unlabeled-*
- 配置名称: ach_tts（阿乔利语文本转语音）
数据文件:
- 划分集: 训练集（train）
路径: data/TTS/ach/ach-train-*
- 划分集: 验证集（validation）
路径: data/TTS/ach/ach-validation-*
- 划分集: 测试集（test）
路径: data/TTS/ach/ach-test-*
- 配置名称: aka_asr（阿坎语自动语音识别）
数据文件:
- 划分集: 训练集（train）
路径: data/ASR/aka/aka-train-*
- 划分集: 验证集（validation）
路径: data/ASR/aka/aka-validation-*
- 划分集: 测试集（test）
路径: data/ASR/aka/aka-test-*
- 划分集: 无标注集（unlabeled）
路径: data/ASR/aka/aka-unlabeled-*
# 其余配置项因篇幅省略，保留原始格式
数据集信息:
- 配置名称: ach_asr（阿乔利语自动语音识别）
特征:
- 字段名: id
数据类型: 字符串
- 字段名: speaker_id
数据类型: 字符串
- 字段名: transcription
数据类型: 字符串
- 字段名: language
数据类型: 字符串
- 字段名: gender
数据类型: 字符串
- 字段名: audio
数据类型: 音频（audio）
# 其余数据集信息因篇幅省略，保留原始格式
# Waxal 数据集
WAXAL数据集是面向非洲语言的大规模多语言语音语料库，相关论文为《WAXAL: A Large-Scale Multilingual African Language Speech Corpus》，链接为https://huggingface.co/papers/2602.02734。

## 目录
- [数据集描述](#dataset-description)
- [自动语音识别数据集](#asr-dataset)
- [文本转语音数据集](#tts-dataset)
- [使用方法](#how-to-use)
- [数据集结构](#dataset-structure)
- [自动语音识别数据字段](#asr-data-fields)
- [文本转语音数据字段](#tts-data-fields)
- [数据划分](#data-splits)
- [数据集构建](#dataset-curation)
- [数据使用注意事项](#considerations-for-using-the-data)
- [补充信息](#additional-information)
- [引用](#citation)

## 数据集描述
Waxal项目提供面向非洲语言的自动语音识别（ASR）和文本转语音（TTS）数据集。本数据集的创建与发布旨在推动提升这些资源匮乏语言的语音与语言技术的准确性和流畅性，并作为数字保存的知识库。

Waxal数据集通过与马克雷雷大学、加纳大学、Digital Umuganda、Media Trust、Loud and Clear以及塞内加尔AIMS的合作获取。本数据集的采集由谷歌与盖茨基金会资助，协议约定数据集需开源公开。塞内加尔语种（沃洛夫语和富拉语）由塞内加尔AIMS提供。

### 自动语音识别数据集
Waxal自动语音识别数据集包含19种非洲语言的数据，涵盖约1250小时的多说话人自然语音转录数据。这19种语言覆盖撒哈拉以南非洲40个国家的超1亿使用者。

| 合作机构 | 语言 | 许可证
:------------------ | :--------------------------------------- | :------------:
马克雷雷大学 | 阿乔利语、卢干达语、马萨巴语、恩扬科勒语、索加语 | CC-BY-SA-4.0
加纳大学 | 阿坎语、埃维语、达格班尼语、达加雷语、伊克波索语 | CC-BY-4.0
Digital Umuganda | 富拉语、林加拉语、绍纳语、马达加斯加语、阿姆哈拉语、奥罗莫语、西达马语、提格雷尼亚语、沃莱塔语 | CC-BY-SA-4.0

### 文本转语音数据集
Waxal文本转语音数据集包含17种非洲语言的文本转语音数据，涵盖超过180小时的高质量单说话人朗读语音数据，朗读内容为语音平衡脚本。

| 合作机构 | 语言 | 许可证
:------------------ | :----------------------------------- | :------------:
马克雷雷大学 | 阿乔利语、卢干达语、斯瓦西里语、恩扬科勒语 | CC-BY-SA-4.0
加纳大学 | 阿坎语（芳蒂语、特威语）、鲍勒语、埃维语 | CC-BY-4.0
Media Trust | 富拉语、伊博语、豪萨语、约鲁巴语、尼日利亚皮钦语 | CC-BY-SA-4.0
Loud and Clear | 基库尤语、卢干达语、卢奥语、斯瓦西里语 | CC-BY-SA-4.0
塞内加尔AIMS | 班巴拉语、普拉尔语、沃洛夫语 | CC-BY-SA-4.0

### 使用方法
使用`datasets`库可以在纯Python环境中规模化加载和预处理数据集。

首先，请确保安装了处理音频数据所需的依赖。需在系统中安装`ffmpeg`。

**Google Colab / Ubuntu**
bash
sudo apt-get install ffmpeg
pip install datasets[audio]

**macOS**
bash
brew install ffmpeg
pip install datasets[audio]

**Windows**
从[ffmpeg.org](https://ffmpeg.org/download.html)下载并安装，并确保其已添加至系统PATH。
bash
pip install datasets[audio]

如果遇到`RuntimeError: Could not load libtorchcodec，请确保`ffmpeg`安装正确，或检查`torch`、`torchaudio`与`torchcodec`版本的兼容性。

**加载自动语音识别数据
如需加载特定语言的自动语音识别数据，请指定配置名称，例如使用`sna_asr`加载绍纳语自动语音识别数据。
python
from datasets import load_dataset, Audio

# 加载绍纳语（sna）自动语音识别数据集
asr_data = load_dataset("google/WaxalNLP", "sna_asr")

# 访问数据划分
train = asr_data['train']
val = asr_data['validation']
test = asr_data['test']

# 示例：访问音频字节与其他字段
example = train[0]
print(f"转录文本：{example['transcription']}")
print(f"采样率：{example['audio']['sampling_rate']}")
# 'array'包含解码后的音频字节为numpy数组
print(f"音频数组形状：{example['audio']['array'].shape}")

**加载文本转语音数据
如需加载特定语言的文本转语音数据，请指定配置名称，例如使用`swa_tts`加载斯瓦西里语文本转语音数据。
python
from datasets import load_dataset

# 加载斯瓦西里语（swa）文本转语音数据集
tts_data = load_dataset("google/WaxalNLP", "swa_tts")

# 访问数据划分
train = tts_data['train']

## 数据集结构
### 自动语音识别数据字段
python
{
'id': 'sna_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'transcription': '...',
'language': 'sna',
'gender': 'Female',
}

* **id**：唯一标识符。
* **speaker_id**：说话人的唯一标识符。
* **audio**：音频数据。
* **transcription**：音频的转录文本。
* **language**：ISO 639-2语言代码。
* **gender**：说话人性别（'Male'为男性，'Female'为女性，或为空。

### 文本转语音数据字段
python
{
'id': 'swa_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'text': '...',
'locale': 'swa',
'gender': 'Female',
}

* **id**：唯一标识符。
* **speaker_id**：说话人的唯一标识符。
* **audio**：音频数据。
* **text**：文本脚本。
* **locale**：ISO 639-2语言代码。
* **gender**：说话人性别。

### 数据划分
对于**自动语音识别数据集**，带转录的数据集划分如下：
* **train**：80%的标注数据。
* **validation**：10%的标注数据。
* **test**：10%的标注数据。
**unlabeled**包含所有无对应转录的样本。
**文本转语音数据集**采用类似结构，数据划分为`train`、`validation`和`test`。

## 数据集构建
本数据集由多个合作机构采集：
| 合作机构 | 数据集 | 许可证
:------------------ | :------------------------------------------------------- | :------
加纳大学 | [UGSpeechData](https://doi.org/10.57760/sciencedb.22298) | CC BY 4.0
Digital Umuganda | [AfriVoice](DigitalUmuganda/AfriVoice) | CC-BY-SA 4.0
马克雷雷大学 | [Yogera Dataset](https://doi.org/10.7910/DVN/BEROE0) | CC-BY-SA 4.0
Media Trust | | CC-BY-SA 4.0
Loud and Clear | | CC-BY-SA 4.0
塞内加尔AIMS | | CC-BY-SA 4.0

## 数据使用注意事项
请检查您所使用的特定语言的许可证，不同合作机构的许可证可能存在差异。

**所属机构**：谷歌研究院

## 版本与维护
- **当前版本**：2.0.0
- **最后更新**：2026年3月

## 引用
bibtex
@article{waxal2026,
title={WAXAL: A Large-Scale Multilingual African Language Speech Corpus},
author={Anonymous},
journal={arXiv preprint arXiv:2602.02734},
year={2026}
}