rishchen/ukrainian-tts-audiobook-pani-nina-parquet-old

--- license: cc-by-nc-sa-4.0 task_categories: - text-to-speech language: - uk size_categories: - 100K<n<1M --- # Ukrainian TTS audiobook dataset (Parquet) Segmented Ukrainian audiobook speech with aligned text, prepared for training and evaluating Text-to-Speech (TTS) models. The dataset is published as Hugging Face-compatible Parquet shards so the Hub **Dataset Preview** can render an `audio` column. That was mabe by using `whisper` (https://github.com/openai/whisper) and `ffmpeg` (https://www.ffmpeg.org/), where with whisper we set start and end of voices + transcribe it and using ffmpeg slice into ~2-10 seconds pieces. ## Motivation / use case - Train Ukrainian TTS / speech synthesis models on long-form narrated speech. - Provide a simple tabular format (`audio` + `text` + metadata) that works well with `datasets`. ## Dataset format Each example is one utterance: - `id` (`int64`): sequential index (0..N-1) - `path` (`string`): relative path of the original `.wav` (kept for traceability) - `audio` (`Audio`): Hugging Face audio feature stored as a struct `{bytes, path}` - `text` (`string`): Ukrainian transcript - `duration` (`float32`): seconds - `source` (`string`): original source recording name (from `metadata.jsonl`) ## Dataset stats - Rows: `116,575` - Total duration: `~114.9 hours` - Sources: `39` unique recordings (see `source` field) - Audio: mono, PCM16, 16 kHz WAV ## Install deps ```bash python -m pip install huggingface_hub==0.30.2 datasets==3.5.0 ``` ## Load with `datasets` From the Hub: ```python from datasets import load_dataset ds = load_dataset("rishchen/ukrainian-tts-audiobook-pani-nina-parquet", split="train") ``` From local Parquet shards: ```python from datasets import load_dataset ds = load_dataset("parquet", data_files={"train": "train/train-*.parquet"})["train"] ``` ## Download from Hugging Face This repo is set up to upload the contents of `train/` as the dataset repository root (so `wavs/...` paths match the Parquet). ```bash export HF_TOKEN="hf_..." python download.py python unpack.py ``` ## Notes for Hub Dataset Preview - The Parquet shards include Hugging Face `datasets` feature metadata so `audio` is recognized as an `Audio` column. - If you generate *path-only* Parquet (default), keep the referenced `.wav` files in the same repo so `audio.path` resolves. - If you generate Parquet with `--embed-audio-bytes`, the dataset is self-contained and you can upload only Parquet. - If you publish both `metadata.jsonl` and Parquet, the Hub may try to load either format; if you want Parquet-only, upload only the shards (or rename/move the raw JSONL out of the dataset root before pushing).许可证：知识共享署名-非商业性使用-相同方式共享4.0（CC BY-NC-SA 4.0） 任务类别： - 文本转语音（Text-to-Speech, TTS） 语言： - 乌克兰语（uk） 规模类别： - 10万 < 样本数 < 100万 # 乌克兰语TTS有声书数据集（Parquet格式） 本数据集为分段式乌克兰语有声书语音数据，附带对齐后的转录文本，专为训练与评估文本转语音（Text-to-Speech, TTS）模型而构建。数据集以适配Hugging Face的Parquet分片形式发布，可使Hub**数据集预览功能**正常渲染`audio`列。 本数据集借助OpenAI的`whisper`（https://github.com/openai/whisper）与`ffmpeg`（https://www.ffmpeg.org/）制作完成：通过`whisper`识别语音片段的起止时刻并完成转录，再使用`ffmpeg`将音频切割为约2至10秒的短片段。 ## 应用场景与研发动机 - 针对长篇章旁白语音，训练乌克兰语TTS/语音合成模型； - 提供简洁的表格式数据格式（包含`audio`、`text`与元数据字段），可与`datasets`库完美适配。 ## 数据集格式 每条数据对应一段独立语音片段： - `id`（`int64`类型）：连续索引，取值范围为0至N-1 - `path`（`string`类型）：原始`.wav`音频文件的相对路径，保留该字段以方便数据溯源 - `audio`（`Audio`类型）：以结构体`{bytes, path}`形式存储的Hugging Face音频特征 - `text`（`string`类型）：乌克兰语语音转录文本 - `duration`（`float32`类型）：语音片段时长，单位为秒 - `source`（`string`类型）：原始录音的来源名称，该信息来自`metadata.jsonl`文件 ## 数据集统计信息 - 总数据条数：116,575 - 总音频时长：约114.9小时 - 来源数量：共39个独立录音，详见`source`字段 - 音频参数：单声道、PCM16编码、16kHz采样率的WAV格式音频 ## 依赖安装 bash python -m pip install huggingface_hub==0.30.2 datasets==3.5.0 ## 通过`datasets`库加载数据集 从Hugging Face Hub加载： python from datasets import load_dataset ds = load_dataset("rishchen/ukrainian-tts-audiobook-pani-nina-parquet", split="train") 从本地Parquet分片加载： python from datasets import load_dataset ds = load_dataset("parquet", data_files={"train": "train/train-*.parquet"})["train"] ## 从Hugging Face下载数据集 本仓库将`train/`目录下的内容设置为数据集根目录，因此`wavs/...`路径可与Parquet文件中的引用路径保持匹配。 bash export HF_TOKEN="hf_..." python download.py python unpack.py ## Hub数据集预览注意事项 - Parquet分片已包含Hugging Face `datasets`库的特征元数据，因此`audio`列会被自动识别为`Audio`类型； - 若仅生成仅含路径引用的Parquet文件（默认设置），请确保所引用的`.wav`文件与Parquet文件处于同一仓库中，以保证`audio.path`可正常解析； - 若通过`--embed-audio-bytes`参数生成Parquet文件，则数据集为自包含格式，仅上传Parquet分片即可； - 若同时上传`metadata.jsonl`与Parquet文件，Hub可能会尝试加载任意一种格式；若仅需使用Parquet格式，请仅上传分片文件，或在推送前将原始JSONL文件移出数据集根目录。