Phsntom/WaxalNLP

---
license:
- cc-by-sa-4.0
- cc-by-4.0
annotation_creators:
- human-annotated
- crowdsourced
language_creators:
- creator_1
tags:
- audio
- automatic-speech-recognition
- text-to-speech
language:
- ach
- aka
- amh
- dag
- dga
- ewe
- fat
- ful
- hau
- ibo
- kik
- kpo
- lin
- lug
- luo
- mas
- mlg
- nyn
- orm
- sid
- sna
- sog
- swa
- tir
- twi
- wal
- yor
multilinguality:
- multilingual
pretty_name: Waxal NLP Datasets
task_categories:
- automatic-speech-recognition
- text-to-speech
source_datasets:
- UGSpeechData
- DigitalUmuganda/AfriVoice
- original
configs:
- config_name: ach_asr
data_files:
- split: train
path: "data/ASR/ach/ach-train-*"
- split: validation
path: "data/ASR/ach/ach-validation-*"
- split: test
path: "data/ASR/ach/ach-test-*"
- split: unlabeled
path: "data/ASR/ach/ach-unlabeled-*"
- config_name: ach_tts
data_files:
- split: train
path: "data/TTS/ach/ach-train-*"
- split: validation
path: "data/TTS/ach/ach-validation-*"
- split: test
path: "data/TTS/ach/ach-test-*"
- config_name: aka_asr
data_files:
- split: train
path: "data/ASR/aka/aka-train-*"
- split: validation
path: "data/ASR/aka/aka-validation-*"
- split: test
path: "data/ASR/aka/aka-test-*"
- split: unlabeled
path: "data/ASR/aka/aka-unlabeled-*"
- config_name: amh_asr
data_files:
- split: train
path: "data/ASR/amh/amh-train-*"
- split: validation
path: "data/ASR/amh/amh-validation-*"
- split: test
path: "data/ASR/amh/amh-test-*"
- split: unlabeled
path: "data/ASR/amh/amh-unlabeled-*"

- config_name: dag_asr
data_files:
- split: train
path: "data/ASR/dag/dag-train-*"
- split: validation
path: "data/ASR/dag/dag-validation-*"
- split: test
path: "data/ASR/dag/dag-test-*"
- split: unlabeled
path: "data/ASR/dag/dag-unlabeled-*"
- config_name: dga_asr
data_files:
- split: train
path: "data/ASR/dga/dga-train-*"
- split: validation
path: "data/ASR/dga/dga-validation-*"
- split: test
path: "data/ASR/dga/dga-test-*"
- split: unlabeled
path: "data/ASR/dga/dga-unlabeled-*"
- config_name: ewe_asr
data_files:
- split: train
path: "data/ASR/ewe/ewe-train-*"
- split: validation
path: "data/ASR/ewe/ewe-validation-*"
- split: test
path: "data/ASR/ewe/ewe-test-*"
- split: unlabeled
path: "data/ASR/ewe/ewe-unlabeled-*"
- config_name: fat_tts
data_files:
- split: train
path: "data/TTS/fat/fat-train-*"
- split: validation
path: "data/TTS/fat/fat-validation-*"
- split: test
path: "data/TTS/fat/fat-test-*"
- config_name: ful_asr
data_files:
- split: train
path: "data/ASR/ful/ful-train-*"
- split: validation
path: "data/ASR/ful/ful-validation-*"
- split: test
path: "data/ASR/ful/ful-test-*"
- split: unlabeled
path: "data/ASR/ful/ful-unlabeled-*"
- config_name: ful_tts
data_files:
- split: train
path: "data/TTS/ful/ful-train-*"
- split: validation
path: "data/TTS/ful/ful-validation-*"
- split: test
path: "data/TTS/ful/ful-test-*"
- config_name: hau_tts
data_files:
- split: train
path: "data/TTS/hau/hau-train-*"
- split: validation
path: "data/TTS/hau/hau-validation-*"
- split: test
path: "data/TTS/hau/hau-test-*"
- config_name: ibo_tts
data_files:
- split: train
path: "data/TTS/ibo/ibo-train-*"
- split: validation
path: "data/TTS/ibo/ibo-validation-*"
- split: test
path: "data/TTS/ibo/ibo-test-*"
- config_name: kik_tts
data_files:
- split: train
path: "data/TTS/kik/kik-train-*"
- split: validation
path: "data/TTS/kik/kik-validation-*"
- split: test
path: "data/TTS/kik/kik-test-*"
- config_name: kpo_asr
data_files:
- split: train
path: "data/ASR/kpo/kpo-train-*"
- split: validation
path: "data/ASR/kpo/kpo-validation-*"
- split: test
path: "data/ASR/kpo/kpo-test-*"
- split: unlabeled
path: "data/ASR/kpo/kpo-unlabeled-*"
- config_name: lin_asr
data_files:
- split: train
path: "data/ASR/lin/lin-train-*"
- split: validation
path: "data/ASR/lin/lin-validation-*"
- split: test
path: "data/ASR/lin/lin-test-*"
- split: unlabeled
path: "data/ASR/lin/lin-unlabeled-*"
- config_name: lug_asr
data_files:
- split: train
path: "data/ASR/lug/lug-train-*"
- split: validation
path: "data/ASR/lug/lug-validation-*"
- split: test
path: "data/ASR/lug/lug-test-*"
- split: unlabeled
path: "data/ASR/lug/lug-unlabeled-*"
- config_name: lug_tts
data_files:
- split: train
path: "data/TTS/lug/lug-train-*"
- split: validation
path: "data/TTS/lug/lug-validation-*"
- split: test
path: "data/TTS/lug/lug-test-*"
- config_name: luo_tts
data_files:
- split: train
path: "data/TTS/luo/luo-train-*"
- split: validation
path: "data/TTS/luo/luo-validation-*"
- split: test
path: "data/TTS/luo/luo-test-*"
- config_name: mas_asr
data_files:
- split: train
path: "data/ASR/mas/mas-train-*"
- split: validation
path: "data/ASR/mas/mas-validation-*"
- split: test
path: "data/ASR/mas/mas-test-*"
- split: unlabeled
path: "data/ASR/mas/mas-unlabeled-*"
- config_name: mlg_asr
data_files:
- split: train
path: "data/ASR/mlg/mlg-train-*"
- split: validation
path: "data/ASR/mlg/mlg-validation-*"
- split: test
path: "data/ASR/mlg/mlg-test-*"
- split: unlabeled
path: "data/ASR/mlg/mlg-unlabeled-*"
- config_name: nyn_asr
data_files:
- split: train
path: "data/ASR/nyn/nyn-train-*"
- split: validation
path: "data/ASR/nyn/nyn-validation-*"
- split: test
path: "data/ASR/nyn/nyn-test-*"
- split: unlabeled
path: "data/ASR/nyn/nyn-unlabeled-*"
- config_name: nyn_tts
data_files:
- split: train
path: "data/TTS/nyn/nyn-train-*"
- split: validation
path: "data/TTS/nyn/nyn-validation-*"
- split: test
path: "data/TTS/nyn/nyn-test-*"
- config_name: orm_asr
data_files:
- split: train
path: "data/ASR/orm/orm-train-*"
- split: validation
path: "data/ASR/orm/orm-validation-*"
- split: test
path: "data/ASR/orm/orm-test-*"
- split: unlabeled
path: "data/ASR/orm/orm-unlabeled-*"

- config_name: sid_asr
data_files:
- split: train
path: "data/ASR/sid/sid-train-*"
- split: validation
path: "data/ASR/sid/sid-validation-*"
- split: test
path: "data/ASR/sid/sid-test-*"
- split: unlabeled
path: "data/ASR/sid/sid-unlabeled-*"

- config_name: sna_asr
data_files:
- split: train
path: "data/ASR/sna/sna-train-*"
- split: validation
path: "data/ASR/sna/sna-validation-*"
- split: test
path: "data/ASR/sna/sna-test-*"
- split: unlabeled
path: "data/ASR/sna/sna-unlabeled-*"
- config_name: tir_asr
data_files:
- split: train
path: "data/ASR/tir/tir-train-*"
- split: validation
path: "data/ASR/tir/tir-validation-*"
- split: test
path: "data/ASR/tir/tir-test-*"
- split: unlabeled
path: "data/ASR/tir/tir-unlabeled-*"

- config_name: sog_asr
data_files:
- split: train
path: "data/ASR/sog/sog-train-*"
- split: validation
path: "data/ASR/sog/sog-validation-*"
- split: test
path: "data/ASR/sog/sog-test-*"
- split: unlabeled
path: "data/ASR/sog/sog-unlabeled-*"
- config_name: swa_tts
data_files:
- split: train
path: "data/TTS/swa/swa-train-*"
- split: validation
path: "data/TTS/swa/swa-validation-*"
- split: test
path: "data/TTS/swa/swa-test-*"
- config_name: twi_tts
data_files:
- split: train
path: "data/TTS/twi/twi-train-*"
- split: validation
path: "data/TTS/twi/twi-validation-*"
- split: test
path: "data/TTS/twi/twi-test-*"
- config_name: yor_tts
data_files:
- split: train
path: "data/TTS/yor/yor-train-*"
- split: validation
path: "data/TTS/yor/yor-validation-*"
- split: test
path: "data/TTS/yor/yor-test-*"
- config_name: wal_asr
data_files:
- split: train
path: "data/ASR/wal/wal-train-*"
- split: validation
path: "data/ASR/wal/wal-validation-*"
- split: test
path: "data/ASR/wal/wal-test-*"
- split: unlabeled
path: "data/ASR/wal/wal-unlabeled-*"

dataset_info:
- config_name: ach_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ach_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: aka_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio

- config_name: dag_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: dga_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ewe_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: fat_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ful_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ful_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: hau_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: ibo_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: kik_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: kpo_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lin_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lug_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: lug_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: luo_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: mas_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: mlg_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: nyn_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: nyn_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio

- config_name: sna_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: sog_asr
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: transcription
dtype: string
- name: language
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio

- config_name: swa_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
- config_name: twi_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio

- config_name: yor_tts
features:
- name: id
dtype: string
- name: speaker_id
dtype: string
- name: text
dtype: string
- name: locale
dtype: string
- name: gender
dtype: string
- name: audio
dtype: audio
---

# Waxal Datasets

## Table of Contents

- [Dataset Description](#dataset-description)
- [ASR Dataset](#asr-dataset)
- [TTS Dataset](#tts-dataset)
- [How to Use](#how-to-use)
- [Dataset Structure](#dataset-structure)
- [ASR Data Fields](#asr-data-fields)
- [TTS Data Fields](#tts-data-fields)
- [Data Splits](#data-splits)
- [Dataset Curation](#dataset-curation)
- [Considerations for Using the Data](#considerations-for-using-the-data)
- [Additional Information](#additional-information)

## Dataset Description

The Waxal project provides datasets for both Automated Speech Recognition (ASR)
and Text-to-Speech (TTS) for African languages. The goal of this dataset's
creation and release is to facilitate research that improves the accuracy and
fluency of speech and language technology for these underserved languages, and
to serve as a repository for digital preservation.

The Waxal datasets are collections acquired through partnerships with Makerere
University, The University of Ghana, Digital Umuganda, and Media Trust.
Acquisition was funded by Google and the Gates Foundation under an agreement to
make the dataset openly accessible.

### ASR Dataset

The Waxal ASR dataset is a collection of data in 19 African languages. It
consists of approximately 1,250 hours of transcribed natural speech from a wide
variety of voices. The 19 languages in this dataset represent over 100 million
speakers across 40 Sub-Saharan African countries.

Provider | Languages | License
:------------------ | :--------------------------------------- | :------------:
Makerere University | Acholi, Luganda, Masaaba, Nyankole, Soga | `CC-BY-SA-4.0`
University of Ghana | Akan, Ewe, Dagbani, Dagaare, Ikposo | `CC-BY-4.0`
Digital Umuganda | Fula, Lingala, Shona, Malagasy, Amharic, Oromo, Sidama, Tigrinya, Wolaytta | `CC-BY-SA-4.0`

### TTS Dataset

The Waxal TTS dataset is a collection of text-to-speech data in 16 African
languages. It consists of approximately 240 hours of scripted natural speech
from a wide variety of voices.

Provider | Languages | License
:------------------ | :----------------------------------- | :------------:
Makerere University | Acholi, Luganda, Kiswahili, Nyankole | `CC-BY-SA-4.0`
University of Ghana | Akan (Fante, Twi) | `CC-BY-4.0`
Media Trust | Fula, Igbo, Hausa, Yoruba | `CC-BY-SA-4.0`

Loud and Clear | Kikuyu, Luo | `CC-BY-SA-4.0`

### How to Use

The `datasets` library allows you to load and pre-process your dataset in pure
Python, at scale.

First, ensure you have the necessary dependencies installed to handle audio
data. You will need `ffmpeg` installed on your system.

**Google Colab / Ubuntu**
```bash
sudo apt-get install ffmpeg
pip install datasets[audio]
```

**macOS**
```bash
brew install ffmpeg
pip install datasets[audio]
```

**Windows**
Download and install from [ffmpeg.org](https://ffmpeg.org/download.html) and ensure it's in your PATH.
```bash
pip install datasets[audio]
```

If you encounter `RuntimeError: Could not load libtorchcodec`, please ensure `ffmpeg` is correctly installed or check for compatibility between your `torch`, `torchaudio`, and `torchcodec` versions.

**Loading ASR Data**

To load ASR data for a specific language, specify the configuration name, e.g.
`sna_asr` for Shona ASR data.

```python
from datasets import load_dataset, Audio

# Load Shona (sna) ASR dataset
asr_data = load_dataset("google/WaxalNLP", "sna_asr")

# Access splits
train = asr_data['train']
val = asr_data['validation']
test = asr_data['test']

# Example: Accessing audio bytes and other fields
example = train[0]
print(f"Transcription: {example['transcription']}")
print(f"Sampling Rate: {example['audio']['sampling_rate']}")
# 'array' contains the decoded audio bytes as a numpy array
print(f"Audio Array Shape: {example['audio']['array'].shape}")
```

**Loading TTS Data**

To load TTS data for a specific language, specify the configuration name, e.g.
`swa_tts` for Swahili TTS data.

```python
from datasets import load_dataset

# Load Swahili (swa) TTS dataset
tts_data = load_dataset("google/WaxalNLP", "swa_tts")

# Access splits
train = tts_data['train']
```

## Dataset Structure

### ASR Data Fields

```python
{
'id': 'sna_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'transcription': '...',
'language': 'sna',
'gender': 'Female',
}
```

* **id**: Unique identifier.
* **speaker_id**: Unique identifier for the speaker.
* **audio**: Audio data.
* **transcription**: Transcription of the audio.
* **language**: ISO 639-2 language code.
* **gender**: Speaker gender ('Male', 'Female', or empty).

### TTS Data Fields

```python
{
'id': 'swa_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'text': '...',
'locale': 'swa',
'gender': 'Female',
}
```

* **id**: Unique identifier.
* **speaker_id**: Unique identifier for the speaker.
* **audio**: Audio data.
* **text**: Text script.
* **locale**: ISO 639-2 language code.
* **gender**: Speaker gender.

### Data Splits

For the **ASR Dataset**, the data with transcriptions is split as follows: *
**train**: 80% of labeled data. * **validation**: 10% of labeled data. *
**test**: 10% of labeled data.

The **unlabeled** split contains all samples that do not have a corresponding
transcription.

The **TTS Dataset** follows a similar structure, with data split into `train`,
`validation`, and `test` sets.

## Dataset Curation

The data was gathered by multiple partners:

Provider | Dataset | License
:------------------ | :------------------------------------------------------- | :------
University of Ghana | [UGSpeechData](https://doi.org/10.57760/sciencedb.22298) | `CC BY 4.0`
Digital Umuganda | [AfriVoice](DigitalUmuganda/AfriVoice) | `CC-BY-SA 4.0`
Makerere University | [Yogera Dataset](https://doi.org/10.7910/DVN/BEROE0) | `CC-BY-SA 4.0`
Media Trust | | `CC-BY-SA 4.0`

## Considerations for Using the Data

Please check the license for the specific languages you are using, as they may
differ between providers.

**Affiliation:** Google Research

## Version and Maintenance

- **Current Version:** 1.0.0
- **Last Updated:** 01/2026

### 数据集元数据
license:
- 知识共享署名-相同方式共享4.0（CC-BY-SA-4.0）
- 知识共享署名4.0（CC-BY-4.0）
annotation_creators:
- 人工标注
- 众包标注
language_creators:
- 创建者1
tags:
- 音频
- 自动语音识别（Automatic Speech Recognition, ASR）
- 文本转语音（Text-to-Speech, TTS）
language:
- ach（阿乔利语）
- aka（阿肯语）
- amh（阿姆哈拉语）
- dag（达格班尼语）
- dga（达加雷语）
- ewe（埃维语）
- fat
- ful（富拉语）
- hau（豪萨语）
- ibo（伊博语）
- kik（基库尤语）
- kpo（伊克波索语）
- lin（林加拉语）
- lug（卢干达语）
- luo（卢奥语）
- mas（马萨巴语）
- mlg（马达加斯加语）
- nyn（恩扬科勒语）
- orm（奥罗莫语）
- sid（西达莫语）
- sna（绍纳语）
- sog（索加语）
- swa（斯瓦希里语）
- tir（提格雷尼亚语）
- twi（特威语）
- wal（沃莱塔语）
- yor（约鲁巴语）
multilinguality:
- 多语言
pretty_name: Waxal NLP数据集（Waxal NLP Datasets）
task_categories:
- 自动语音识别（Automatic Speech Recognition, ASR）
- 文本转语音（Text-to-Speech, TTS）
source_datasets:
- UGSpeechData
- DigitalUmuganda/AfriVoice
- 原始数据集
configs:
- config_name: ach_asr（阿乔利语自动语音识别任务）
data_files:
- split: 训练集
path: "data/ASR/ach/ach-train-*"
- split: 验证集
path: "data/ASR/ach/ach-validation-*"
- split: 测试集
path: "data/ASR/ach/ach-test-*"
- split: 未标注集
path: "data/ASR/ach/ach-unlabeled-*"
- config_name: ach_tts（阿乔利语文本转语音任务）
data_files:
- split: 训练集
path: "data/TTS/ach/ach-train-*"
- split: 验证集
path: "data/TTS/ach/ach-validation-*"
- split: 测试集
path: "data/TTS/ach/ach-test-*"
# 其余配置项按此格式类推，此处省略重复内容以节省篇幅
dataset_info:
- config_name: ach_asr（阿乔利语自动语音识别任务）
features:
- name: id
dtype: 字符串
- name: speaker_id
dtype: 字符串
- name: transcription
dtype: 字符串
- name: language
dtype: 字符串
- name: gender
dtype: 字符串
- name: audio
dtype: 音频数据
- config_name: ach_tts（阿乔利语文本转语音任务）
features:
- name: id
dtype: 字符串
- name: speaker_id
dtype: 字符串
- name: text
dtype: 字符串
- name: locale
dtype: 字符串
- name: gender
dtype: 字符串
- name: audio
dtype: 音频数据
# 其余配置项按此格式类推，此处省略重复内容

# Waxal数据集

## 目录
- [数据集描述](#dataset-description)
- [自动语音识别（ASR）数据集](#asr-dataset)
- [文本转语音（TTS）数据集](#tts-dataset)
- [使用方法](#how-to-use)
- [数据集结构](#dataset-structure)
- [ASR数据集数据字段](#asr-data-fields)
- [TTS数据集数据字段](#tts-data-fields)
- [数据拆分](#data-splits)
- [数据集整理](#dataset-curation)
- [数据使用注意事项](#considerations-for-using-the-data)
- [附加信息](#additional-information)

## 数据集描述
Waxal项目为非洲语言提供自动语音识别（Automatic Speech Recognition, ASR）与文本转语音（Text-to-Speech, TTS）两类数据集。本数据集的创建与发布旨在推动相关研究，提升针对这些资源匮乏语言的语音与语言技术的准确率与流畅度，并作为数字保存的知识库。
Waxal数据集由马凯雷雷大学、加纳大学、数字乌穆甘达项目与媒体信托合作采集，该采集项目由谷歌与盖茨基金会资助，并约定数据集需公开可获取。

### 自动语音识别（ASR）数据集
Waxal ASR数据集涵盖19种非洲语言，包含约1250小时来自多样发声群体的转录自然语音数据。本数据集覆盖的19种语言拥有超1亿使用者，分布于撒哈拉以南非洲的40个国家。

| 提供方 | 涵盖语言 | 许可证 |
|:---------------|:-----------------------------------------------------------------------|:--------------------------:|
| 马凯雷雷大学 | 阿乔利语、卢干达语、马萨巴语、恩扬科勒语、索加语 | `CC-BY-SA-4.0` |
| 加纳大学 | 阿肯语、埃维语、达格班尼语、达加雷语、伊克波索语 | `CC-BY-4.0` |
| 数字乌穆甘达项目 | 富拉语、林加拉语、绍纳语、马达加斯加语、阿姆哈拉语、奥罗莫语、西达莫语、提格雷尼亚语、沃莱塔语 | `CC-BY-SA-4.0` |

### 文本转语音（TTS）数据集
Waxal TTS数据集涵盖16种非洲语言的文本转语音数据，包含约240小时来自多样发声群体的脚本化自然语音数据。

| 提供方 | 涵盖语言 | 许可证 |
|:---------------|:-----------------------------------------------------------------------|:--------------------------:|
| 马凯雷雷大学 | 阿乔利语、卢干达语、斯瓦希里语、恩扬科勒语 | `CC-BY-SA-4.0` |
| 加纳大学 | 阿肯语（方蒂语、特威语） | `CC-BY-4.0` |
| 媒体信托 | 富拉语、伊博语、豪萨语、约鲁巴语 | `CC-BY-SA-4.0` |
| Loud and Clear | 基库尤语、卢奥语 | `CC-BY-SA-4.0` |

### 使用方法
借助Hugging Face `datasets`库，您可以通过纯Python代码规模化加载并预处理数据集。
首先，请确保安装了处理音频数据所需的依赖项，您需在系统中安装`ffmpeg`。

**Google Colab / Ubuntu 系统**
bash
sudo apt-get install ffmpeg
pip install datasets[audio]

**macOS 系统**
bash
brew install ffmpeg
pip install datasets[audio]

**Windows 系统**
请从[ffmpeg.org](https://ffmpeg.org/download.html)下载并安装`ffmpeg`，并确保其已添加至系统PATH环境变量。
bash
pip install datasets[audio]

若您遇到`RuntimeError: Could not load libtorchcodec`错误，请确认`ffmpeg`已正确安装，或检查`torch`、`torchaudio`与`torchcodec`的版本兼容性。

#### 加载ASR数据集
若需加载特定语言的ASR数据集，请指定配置名称，例如使用`sna_asr`加载绍纳语ASR数据集。
python
from datasets import load_dataset, Audio

# 加载绍纳语（sna）ASR数据集
asr_data = load_dataset("google/WaxalNLP", "sna_asr")

# 访问数据拆分
train = asr_data['train']
val = asr_data['validation']
test = asr_data['test']

# 示例：访问音频数据与其他字段
example = train[0]
print(f"转录文本: {example['transcription']}")
print(f"采样率: {example['audio']['sampling_rate']}")
# 'array' 字段存储了解码为NumPy数组的音频数据
print(f"音频数组形状: {example['audio']['array'].shape}")

#### 加载TTS数据集
若需加载特定语言的TTS数据集，请指定配置名称，例如使用`swa_tts`加载斯瓦希里语TTS数据集。
python
from datasets import load_dataset

# 加载斯瓦希里语（swa）TTS数据集
tts_data = load_dataset("google/WaxalNLP", "swa_tts")

# 访问数据拆分
train = tts_data['train']

## 数据集结构
### ASR数据集数据字段
python
{
'id': 'sna_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'transcription': '...',
'language': 'sna',
'gender': 'Female',
}

各字段说明：
* **id**: 唯一标识符。
* **speaker_id**: 说话人唯一标识符。
* **audio**: 音频数据。
* **transcription**: 语音转录文本。
* **language**: ISO 639-2语言代码。
* **gender**: 说话人性别（可选值为`Male`、`Female`或留空）。

### TTS数据集数据字段
python
{
'id': 'swa_0',
'speaker_id': '...',
'audio': {
'array': [...],
'sample_rate': 16_000
},
'text': '...',
'locale': 'swa',
'gender': 'Female',
}

各字段说明：
* **id**: 唯一标识符。
* **speaker_id**: 说话人唯一标识符。
* **audio**: 音频数据。
* **text**: 文本脚本。
* **locale**: ISO 639-2语言代码。
* **gender**: 说话人性别。

### 数据拆分
对于**ASR数据集**，带转录文本的标注数据拆分规则如下：
* **训练集（train）**: 占标注数据的80%。
* **验证集（validation）**: 占标注数据的10%。
* **测试集（test）**: 占标注数据的10%。
**未标注集（unlabeled）**包含所有无对应转录文本的样本。
**TTS数据集**采用类似的拆分规则，数据被划分为`train`（训练集）、`validation`（验证集）与`test`（测试集）。

## 数据集整理
本数据集由多方合作采集：

| 提供方 | 数据集来源 | 许可证 |
|:---------------|:-----------------------------------------------------------------------|:--------------------------:|
| 加纳大学 | [UGSpeechData](https://doi.org/10.57760/sciencedb.22298) | `CC BY 4.0` |
| 数字乌穆甘达项目 | [AfriVoice](DigitalUmuganda/AfriVoice) | `CC-BY-SA 4.0` |
| 马凯雷雷大学 | [Yogera数据集](https://doi.org/10.7910/DVN/BEROE0) | `CC-BY-SA-4.0` |
| 媒体信托 | 无公开指定数据集 | `CC-BY-SA-4.0` |

## 数据使用注意事项
请确认您所使用的特定语言对应的许可证，不同提供方的许可证可能存在差异。

**所属机构**: 谷歌研究院

## 版本与维护
- **当前版本**: 1.0.0
- **最后更新时间**: 2026年1月