UBC-NLP/afroscope-data

---
dataset_info:
features:
- name: sentence
dtype: string
- name: ISO-639-3
dtype: string
- name: script
dtype: string
- name: domain
dtype: string
- name: lang_family_1
dtype: string
- name: lang_family_2
dtype: string
- name: lang_family_3
dtype: string
- name: lang_family_4
dtype: string
- name: lang_family_5
dtype: string
- name: label
dtype: string
splits:
- name: train
num_bytes: 4035426467
num_examples: 17755776
- name: validation
num_bytes: 237264556
num_examples: 1044489
- name: test
num_bytes: 17240082
num_examples: 65503
download_size: 1339922188
dataset_size: 4289931105
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: validation
path: data/validation-*
- split: test
path: data/test-*
---
# Afroscope-Data

This is the corpus released as part of the AfroScope project for large-scale African language identification, supporting 713 languages. It provides sentence-level text with language labels and linguistic metadata and was used to train the open [Afroscope-model](https://huggingface.co/14kwonss/afroscope-model).

The source project and additional resources are openly available here:

https://github.com/skwon01-UBC/AfroScope

## Dataset Features

- **sentence** *(string)*: The text sample.
- **ISO-639-3** *(string)*: The ISO 639-3 language code.
- **script** *(string)*: The writing script (e.g., `Latn`, `Arab`).
- **domain** *(string)*: Source/domain category of the sample.
- **lang_family_1 ~ 5** *(string)*: Language family hierarchy.
- **label** *(string)*: The final label used for classification.

## Loading the dataset

```python
from datasets import load_dataset

ds = load_dataset(
"UBC-NLP/afroscope-dataset",
split="train"
)

print(ds[0])
```

## Citation

```bibtex
@article{kwon2026afroscope,
title={AfroScope: A Framework for Studying the Linguistic Landscape of Africa},
author={Kwon, Sang Yun and Elmadany, AbdelRahim and Abdul-Mageed, Muhammad},
journal={arXiv preprint arXiv:2601.13346},
year={2026}
}
```