mnazari/nena_speech_1_0_test

Name: mnazari/nena_speech_1_0_test
Creator: mnazari
Published: 2024-03-04T19:01:37+08:00

Hugging Face2023-10-27 更新2024-03-04 收录

多语言语音识别

方言研究

数据链接：

https://hf-mirror.com/datasets/mnazari/nena_speech_1_0_test 数据链接链接失效反馈

官方服务：

资源简介：

--- pretty_name: NENA Speech Dataset 1.0 (test) annotations_creators: - crowdsourced - Geoffrey Khan language_creators: - crowdsourced language: - aii - cld - huy - lsd - trg - aij - bhn - hrt - kqd - syn license: - cc0-1.0 multilinguality: - multilingual task_categories: - automatic-speech-recognition - text-to-speech - translation size_categories: - 10K<n<100K - 1K<n<10K - n<1K --- # Dataset Card for NENA Speech Dataset 1.0 (test) ## Table of Contents - [Dataset Summary](#dataset-summary) - [Dataset Description](#dataset-description) - [Languages](#languages) - [How to Use](#how-to-use) - [Dataset Structure](#dataset-structure) - [Data Instances](#data-instances) - [Data Fields](#data-fields) - [Data Splits](#data-splits) - [Dataset Creation](#dataset-creation)  - [Building the Dataset](#building-the-dataset) - [Personal and Sensitive Information](#personal-and-sensitive-information)  - [Additional Information](#additional-information) - [Licensing Information](#licensing-information) - [Citation Information](#citation-information) ## ⚠️ This is a temperary repository that will be replaced by end of 2023 ## Dataset Summary NENA Speech is a multimodal dataset to help teach machines how real people speak the Northeastern Neo-Aramaic (NENA) dialects. The NENA dialects form a very diverse group of Aramaic dialects spoken by Christian and Jewish communities indigenous to northwestern Iran, northern Iraq, and southeastern Türkiye. NENA Speech consists of multimodal examples of speech of the NENA dialects. While all documented NENA dialects are included, not all have data yet, and some will never due to recent loss of their final speakers. ## Dataset Description - **Homepage**: https://crowdsource.nenadb.dev/ - **Point of Contact:** [Matthew Nazari](mailto:matthewnazari@college.harvard.edu) ## Languages The NENA dialects form a very diverse group of Aramaic dialects spoken by Christian and Jewish communities indigenous to northwestern Iran, northern Iraq, and southeastern Türkiye. Speakers of the Christian dialects call their language Assyrian and Chaldean in English. In their language these speakers use multiple different terms (e.g. suráy, sureth, ḥadiṯan, senaya). Speakers of the Jewish dialects call their language lišana deni, lišanət noshan, lišana nosha, lišana didan, all meaning "our language". Some names reflect the consciousness of it being a specifically Jewish language (e.g. lišan hozaye, hulaula). NENA Speech has a subset for all of the over 150 NENA dialects. Not all dialects have examples available yet. Some dialects will never have examples available due to the loss of their final speakers in recent years. ## How to Use The `datasets` library allows you to load and pre-process your dataset in pure Python, at scale. The dataset can be downloaded and prepared in one call to your local drive by using the `load_dataset` function. For example, simply specify the corresponding language config name (e.g., "urmi (christian)" for the dialect of the Assyrian Christians of Urmi): ```python from datasets import load_dataset nena_speech = load_dataset("mnazari/nena_speech_1_0_test", "urmi (christian)", split="train") ``` To find out more about loading and preparing audio datasets, head over to [hf.co/blog/audio-datasets](https://huggingface.co/blog/audio-datasets). ## Dataset Structure ### Data Instances The NENA Speech dataset is a multimodal dataset that consists of three different kinds of examples: 1. **Unlabeled speech examples:** these contain audio of speech (`audio`) but no accompanying transcription (`transcription`) or translation (`translation`). This is useful for representation learning. 2. **Transcribed speech examples:** these contain both audio and transcription of speech. These are useful for machine learning tasks like automatic speech recognition and speech synthesis. 3. **Transcribed and translated speech examples:** these kinds of examples contain audio, transcription, and translation of speech. These are useful for tasks like multimodal translation. Make sure to filter for the kinds of examples you need for your task before before using it. ```json { "transcription": "gu-mdìta.ˈ", "translation": "in the town.", "audio": { "path": "et/train/nena_speech_0uk14ofpom196aj.mp3", "array": array([-0.00048828, -0.00018311, -0.00137329, ..., 0.00079346, 0.00091553, 0.00085449], dtype=float32), "sampling_rate": 48000 }, "locale": "IRN", "proficiency": "proficient as mom", "age": "70's", "crowdsourced": true, "unlabeled": true, "interrupted": true, "client_id": "gwurt1g1ln" , "path": "et/train/nena_speech_0uk14ofpom196aj.mp3", } ``` ### Data Fields - `transcription (string)`: The transcription of what was spoken (e.g. `"beta"`) - `translation (string)`: The translation of what was spoken in English (e.g. `"house"`) - `audio (dict)`: A dictionary containing the path to the downloaded audio file, the decoded audio array, and the sampling rate. Note that when accessing the audio column: `dataset[0]["audio"]` the audio file is automatically decoded and resampled to `dataset.features["audio"].sampling_rate`. Decoding and resampling of a large number of audio files might take a significant amount of time. Thus it is important to first query the sample index before the "audio" column, i.e. `dataset[0]["audio"]` should always be preferred over `dataset["audio"][0]`. - `locale (string)`: The locale of the speaker - `proficiency (string)`: The proficiency of the speaker - `age (string)`: The age of the speaker (e.g. `"20's"`, `"50's"`, `"100+"`) - `crowdsourced (bool)`: Indicates whether the example was crowdsourced as opposed to collected from existing language documentation resources - `interrupted (bool)`: Indicates whether the example was interrupted with the speaker making sound effects or switching into another language - `client_id (string)`: An id for which client (voice) made the recording - `path (string)`: The path to the audio file ### Data Splits The examples have been subdivided into three portions: 1. **dev:** the validation split (10%) 3. **test:** the test split (10%) 2. **train:** the train split (80%) The dev, test, train are all data that has been reviewed, deemed of high quality and split into dev, test and train. ## Dataset Creation  ### Building the Dataset The NENA Speech dataset itself is built using `build.py`. First, install the necessary requirements. ``` pip install -r requirements.txt ``` Next, build the dataset. ``` python build.py --build ``` Finally, push to the HuggingFace dataset repository. ## Personal and Sensitive Information The dataset consists of people who have donated their voice online. You agree to not attempt to determine the identity of speakers in the Common Voice dataset. ## Data Preprocessing The dataset consists of three different kinds of examples (see [Data Instances](#data-instances)). Make sure to filter for the kinds of examples you need for your task before before using it. For example, for automatic speech recognition you will want to filter for examples with transcriptions. In most tasks, you will want to filter out examples that are interrupted (e.g. by the speaker making sound effects, by the speaker switching into a another language). ```python from datasets import load_dataset ds = load_dataset("mnazari/nena_speech_1_0_test", "urmi (christian)", split="train") def filter_for_asr(example): return example['transcription'] and not example['interrupted'] ds = ds.filter(filter_for_asr, desc="filter dataset") ``` Transcriptions include markers of linguistic and acoustic features which may be removed in certain tasks (e.g. word stress, nuclear stress, intonation group markers, vowel length). ```python from datasets import load_dataset ds = load_dataset("mnazari/nena_speech_1_0_test", "urmi (christian)", split="train") def prepare_dataset(batch): chars_to_remove = ['ˈ', '̀', '́', '̄', '̆', '.', ',', '?', '!'] for char in chars_to_remove: batch["transcription"] = batch["transcription"].replace(char, "") return batch ds = ds.map(prepare_dataset, desc="preprocess dataset") ```  ## Additional Information ### Licensing Information Public Domain, [CC-0](https://creativecommons.org/share-your-work/public-domain/cc0/). ### Citation Information This work has not yet been published.

提供机构：

mnazari

原始信息汇总

NENA Speech Dataset 1.0 (test) 数据集概述

数据集摘要

NENA Speech 是一个多模态数据集，旨在帮助机器学习真实人群如何说东北新阿拉姆语（NENA）方言。NENA 方言是由居住在伊朗西北部、伊拉克北部和土耳其东南部的基督教和犹太社区所说的阿拉姆语方言组成的非常多样化的群体。

NENA Speech 包含 NENA 方言的多模态语音示例。虽然所有记录的 NENA 方言都包括在内，但并非所有方言都有数据，由于最近失去了最后的发言人，有些方言将永远不会有数据。

语言

NENA 方言是由基督教和犹太社区所说的阿拉姆语方言组成的非常多样化的群体，主要分布在伊朗西北部、伊拉克北部和土耳其东南部。基督教方言的发言人用英语称他们的语言为亚述语和迦勒底语，而在他们的语言中，这些发言人使用多个不同的术语（例如 suráy, sureth, ḥadiṯan, senaya）。犹太方言的发言人称他们的语言为 lišana deni, lišanət noshan, lišana nosha, lišana didan，意为“我们的语言”。一些名称反映了它作为一种特定犹太语言的意识（例如 lišan hozaye, hulaula）。

NENA Speech 包含超过 150 种 NENA 方言的子集。并非所有方言都有示例可用，有些方言由于最近失去了最后的发言人，将永远不会有示例可用。

数据集结构

数据实例

NENA Speech 数据集是一个多模态数据集，包含三种不同类型的示例：

未标记的语音示例：包含语音的音频（audio），但没有伴随的转录（transcription）或翻译（translation）。这对于表示学习很有用。
转录的语音示例：包含语音的音频和转录。这些对于机器学习任务如自动语音识别和语音合成很有用。
转录和翻译的语音示例：包含语音的音频、转录和翻译。这些对于多模态翻译等任务很有用。

在使用数据集之前，请确保根据您的任务过滤所需的示例类型。

数据字段

transcription (string)：所说的内容的转录（例如 "beta"）
translation (string)：所说的内容在英语中的翻译（例如 "house"）
audio (dict)：包含下载音频文件的路径、解码的音频数组和采样率的字典。注意，当访问音频列时：dataset[0]["audio"]，音频文件会自动解码并重采样到 dataset.features["audio"].sampling_rate。解码和重采样大量音频文件可能需要大量时间。因此，在访问“audio”列之前，应始终先查询样本索引，即 dataset[0]["audio"] 应始终优先于 dataset["audio"][0]。
locale (string)：发言人的地区
proficiency (string)：发言人的熟练程度
age (string)：发言人的年龄（例如 "20s", "50s", "100+"）
crowdsourced (bool)：指示示例是否是通过众包收集的，而不是从现有的语言文档资源中收集的
interrupted (bool)：指示示例是否被发言人发出声音效果或切换到另一种语言打断
client_id (string)：录制音频的客户端（声音）的 ID
path (string)：音频文件的路径

数据分割

示例已被细分为三个部分：

dev：验证集（10%）
test：测试集（10%）
train：训练集（80%）

dev、test 和 train 都是经过审查、被认为是高质量的数据，并被分割成 dev、test 和 train。

数据集创建

构建数据集

NENA Speech 数据集本身是使用 build.py 构建的。

首先，安装必要的依赖项。

bash pip install -r requirements.txt

接下来，构建数据集。

bash python build.py --build

最后，推送到 HuggingFace 数据集仓库。

个人和敏感信息

数据集包含在线捐赠声音的人。您同意不尝试确定 Common Voice 数据集中发言人的身份。

数据预处理

数据集包含三种不同类型的示例（见数据实例）。

在使用数据集之前，请确保根据您的任务过滤所需的示例类型。例如，对于自动语音识别，您需要过滤带有转录的示例。

在大多数任务中，您需要过滤掉被打断的示例（例如，发言人发出声音效果，发言人切换到另一种语言）。

python from datasets import load_dataset

ds = load_dataset("mnazari/nena_speech_1_0_test", "urmi (christian)", split="train")

def filter_for_asr(example): return example[transcription] and not example[interrupted]

ds = ds.filter(filter_for_asr, desc="filter dataset")

转录包括语言和声学特征的标记，这些标记在某些任务中可能会被移除（例如，单词重音、核重音、语调组标记、元音长度）。

python from datasets import load_dataset

ds = load_dataset("mnazari/nena_speech_1_0_test", "urmi (christian)", split="train")

def prepare_dataset(batch): chars_to_remove = [ˈ, ̀, ́, ̄, ̆, ., ,, ?, !] for char in chars_to_remove: batch["transcription"] = batch["transcription"].replace(char, "") return batch

ds = ds.map(prepare_dataset, desc="preprocess dataset")