Massive Arabic Speech Corpus (MASC)

This paper releases and describes the creation of the Massive Arabic Speech Corpus (MASC). This corpus is a dataset that contains 1,000 hours of speech sampled at 16~kHz and crawled from over 700 YouTube channels. MASC is multi-regional, multi-genre, and multi-dialect dataset that is intended to advance the research and development of Arabic speech technology with the special emphasis on Arabic speech recognition. In addition to MASC, a pre-trained 3-gram language model and a pre-trained automatic speech recognition model are also developed and made available for interested researches. For a better language model, a new and unified Arabic speech corpus is required, and thus, a dataset of 12~M unique Arabic words is created and released. To make practical and convenient use of MASC, the whole dataset is stratified based on dialect into clean and noisy portions. Each of the two portions is then stratified and divided into three subsets: development, test, and training sets. The best word error rate achieved by the speech recognition model is 19.8% for the clean development set and 21.8% for the clean test set.

本论文公开并详述了大规模阿拉伯语语音语料库（Massive Arabic Speech Corpus，以下简称MASC）的构建流程。该语料库包含总计1000小时的语音数据，采样率为16kHz，数据源自700余个YouTube频道的爬取内容。MASC是一款多地域、多体裁、多方言的语料库，旨在推动阿拉伯语语音技术的研发进程，尤其侧重阿拉伯语语音识别方向。除公开MASC外，本研究还开发并发布了预训练3-gram语言模型与预训练自动语音识别（Automatic Speech Recognition）模型，供有需求的研究人员使用。为构建更优质的语言模型，研究人员亟需全新的标准化阿拉伯语语音语料库，因此本次工作还创建并公开了包含1200万个唯一阿拉伯语词汇的数据集。为便于实际便捷应用，整个语料库按方言分层划分为纯净语音与含噪语音两个子集。随后，这两个子集又分别划分为开发集、测试集与训练集三个子数据集。该语音识别模型在纯净开发集上的最优词错误率为19.8%，在纯净测试集上的最优词错误率为21.8%。

搜集汇总

背景与挑战

背景概述

MASC是一个大规模的阿拉伯语语音语料库，包含1000小时的16kHz采样语音数据，从700多个YouTube频道爬取，具有多地区、多类型和多方言的特点，专门用于推动阿拉伯语语音识别研究。数据集还提供了预训练的3-gram语言模型和自动语音识别模型，以及一个来自Twitter的12M独特阿拉伯语单词数据集，以增强语言建模能力。整体数据集结构完整，包括音频、字幕和训练评估子集，适用于语音识别模型的训练和评估。