coml/ABX-accent

---
license: mit
---
ABX-accent
-----------

The ABX-accent project is based on the preparation and evaluation of the Accented English Speech Recognition Challenge (AESRC) dataset [1], using fastABX [2] for evaluation. This repository provides all the items files you can use for evaluation.

What is ABX Evaluation?
-----------------------
The ABX metric evaluates whether a representation X of a speech unit (e.g., the triphone “bap”) is closer to a same-category example A (also “bap”) than to a different-category example B (e.g., “bop”). The ABX error rate is calculated by averaging the discrimination errors over all minimal triphone pairs (ie., differing only by the central phoneme) in the corpus.
This benchmark focuses on the more challenging ABX across speaker task, where the X example is spoken by a different speaker than the ones in pair (A, B), testing speaker-invariant phonetic discrimination.

This benchmark focuses on the more challenging ABX across speaker task, where the X example is spoken by a different speaker than the ones in pair (A, B), testing speaker-invariant phonetic discrimination.

About the Dataset.
-----------------------

The **[Accented English Speech Recognition Challenge](https://arxiv.org/abs/2102.10233)** dataset includes recordings from ten different regional accents:
American, British, Canadian, Chinese, Indian, Japanese, Korean, Portuguese, Spanish, Russian.
For academic research only. You can apply this dataset following the instructions on this page: https://www.nexdata.ai/company/sponsored-datasets.

Getting Started
-------------------
To begin working with the AESRC development data and run evaluations, you will find the following resources in the [GitHub repository](https://github.com/bootphon/ABX-accent).
The benchmark is part of the first task of the ZeroSpeech Benchmark on https://zerospeech.com.

References
-----------
- [1] Xian Shi, Fan Yu, Yizhou Lu, Yuhao Liang, Qiangze Feng, Daliang Wang, Yanmin Qian, and Lei Xie, “The accented english speech recognition challenge 2020: open datasets, tracks, baselines, results and methods,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 2021, pp. 6918–6922.

- [2] Maxime Poli, Emmanuel Chemla, Emmanuel Dupoux "fastabx: A library for efficient computation of ABX discriminability" arXiv:2505.02692v1 [cs.CL] 5 May 2025.