rishchen/ukrainian-tts-audiobook-pani-nina-parquet

---
license: cc-by-nc-sa-4.0
task_categories:
- text-to-speech
language:
- uk
size_categories:
- 100K<n<1M
---

# Ukrainian TTS audiobook dataset (Parquet)

Segmented Ukrainian audiobook speech with aligned text, prepared for training and evaluating Text-to-Speech (TTS) models.
The dataset is published as Hugging Face-compatible Parquet shards so the Hub **Dataset Preview** can render an `audio` column.

The dataset was prepared using [whisper](https://github.com/openai/whisper) and [ffmpeg](https://www.ffmpeg.org/):
- Whisper was used for transcription and approximate segment timing.
- FFmpeg was used to slice audio into short utterances (roughly 2-10 seconds).

## Motivation / use case

- Train Ukrainian TTS / speech synthesis models on long-form narrated speech.
- Use a simple tabular format (`audio` + `text` + metadata) that works with `datasets`.
- Keep a reversible pack/unpack workflow for Parquet distribution.

## Dataset format

Each example is one utterance:

- `id` (`int64`): sequential index (0..N-1)
- `path` (`string`): normalized relative path used by HF Audio
- `audio` (`Audio`): Hugging Face audio feature stored as struct `{bytes, path}`
- `original_path` (`string`): exact original path value from source metadata
- `text` (`string`): Ukrainian transcript
- `text_normalaised` (`string`): normalized transcript (if available)
- `text_phonemized` (`string`): phonemized transcript (if available)
- `text_normalaised_phonemized` (`string`): normalized+phonemized transcript (if available)
- `duration` (`float32`): seconds
- `wer` (`float32`): quality proxy from ASR alignment
- `source` (`string`): original source recording name

## Dataset stats

- Rows: `138,447`
- Total duration: `~143.6 hours`
- Sources: `39` unique recordings (see `source` field)
- Audio format: mono, PCM16, 16 kHz WAV

## Install deps

```bash
python -m pip install pyarrow datasets huggingface_hub
```

## Load with huggingface_hub (preferable)

```python
import os
from huggingface_hub import snapshot_download

snapshot_download(
repo_id="rishchen/ukrainian-tts-audiobook-pani-nina-parquet",
repo_type="dataset",
local_dir="hf_parquet",
allow_patterns=["*"],
token=os.getenv("HF_TOKEN"),
)
```

## Load with `datasets`

From the Hub:

```python
from datasets import load_dataset

ds = load_dataset("rishchen/ukrainian-tts-audiobook-pani-nina-parquet", split="train")
```

From local Parquet shards:

```python
from datasets import load_dataset

ds = load_dataset("parquet", data_files={"train": "train_parquet/*.parquet"})["train"]
```

## Pack into Parquet

Default behavior packs wav bytes into parquet (`audio.bytes`) so shards are self-contained:

```bash
python make_hf_parquet.py \
--split-dir train \
--path-key auto \
--rows-per-shard 10000 \
--output-dir train_parquet \
--overwrite
```

If you want path-only parquet (smaller files), disable embedding:

```bash
python make_hf_parquet.py \
--split-dir train \
--path-key auto \
--rows-per-shard 10000 \
--no-embed-audio-bytes \
--output-dir train_parquet \
--overwrite
```

## Unpack from Parquet

This restores audio files and regenerates metadata with matching original path key/value:

```bash
python unpack_hf_parquet.py \
--input-dir train_parquet \
--output-dir train_unpacked \
--overwrite
```

For path-only parquet, also provide original audio root:

```bash
python unpack_hf_parquet.py \
--input-dir train_parquet \
--output-dir train_unpacked \
--source-audio-root train \
--overwrite
```

license: 知识共享署名-非商业性使用-相同方式共享4.0协议（CC-BY-NC-SA-4.0）
task_categories:
- 文本转语音（text-to-speech）
language:
- 乌克兰语（uk）
size_categories:
- 10万至100万条数据（100K<n<1M）
---

# 乌克兰语TTS有声书数据集（Parquet格式）

本数据集包含经文本对齐的分段乌克兰语有声书语音，专为文本转语音（Text-to-Speech, TTS）模型的训练与评估打造。数据集采用兼容Hugging Face的Parquet分片格式发布，以便Hub**数据集预览功能**可渲染`audio`列。

本数据集通过[Whisper](https://github.com/openai/whisper)与[FFmpeg](https://www.ffmpeg.org/)构建：
- 利用Whisper完成转录与近似分段时间戳标注；
- 借助FFmpeg将音频切割为时长约2至10秒的短语音片段。

## 应用场景与设计初衷
- 在长篇章旁白语音数据集上训练乌克兰语文本转语音/语音合成模型；
- 采用适配`datasets`库的简洁表格格式，包含`audio`、`text`与元数据字段；
- 为Parquet格式分发提供可逆的打包/解包工作流。

## 数据集格式
每条样本对应一个语音片段：
- `id`（`int64`类型）：连续索引（取值范围0~N-1）
- `path`（`string`类型）：Hugging Face音频功能所需的标准化相对路径
- `audio`（`Audio`类型）：以结构体`{bytes, path}`存储的Hugging Face音频特征
- `original_path`（`string`类型）：源元数据中记录的原始完整路径
- `text`（`string`类型）：乌克兰语原始转录文本
- `text_normalaised`（`string`类型）：归一化转录文本（如可用）
- `text_phonemized`（`string`类型）：音素化转录文本（如可用）
- `text_normalaised_phonemized`（`string`类型）：归一化且音素化的转录文本（如可用）
- `duration`（`float32`类型）：语音片段时长（单位：秒）
- `wer`（`float32`类型）：自动语音识别（Automatic Speech Recognition, ASR）对齐得到的质量代理指标
- `source`（`string`类型）：原始源录音名称

## 数据集统计信息
- 样本量：138,447条
- 总时长：约143.6小时
- 源文件数量：39个独立录音（详见`source`字段）
- 音频格式：单声道、PCM16编码、16kHz采样率的WAV格式

## 安装依赖
bash
python -m pip install pyarrow datasets huggingface_hub

## 通过Hugging Face Hub加载（推荐方式）
python
import os
from huggingface_hub import snapshot_download

snapshot_download(
repo_id="rishchen/ukrainian-tts-audiobook-pani-nina-parquet",
repo_type="dataset",
local_dir="hf_parquet",
allow_patterns=["*"],
token=os.getenv("HF_TOKEN"),
)

## 通过`datasets`库加载
从Hugging Face Hub加载：
python
from datasets import load_dataset

ds = load_dataset("rishchen/ukrainian-tts-audiobook-pani-nina-parquet", split="train")

从本地Parquet分片加载：
python
from datasets import load_dataset

ds = load_dataset("parquet", data_files={"train": "train_parquet/*.parquet"})["train"]

## 打包为Parquet格式
默认行为会将WAV音频字节嵌入Parquet文件（`audio.bytes`），使分片文件具备自包含性：
bash
python make_hf_parquet.py
--split-dir train
--path-key auto
--rows-per-shard 10000
--output-dir train_parquet
--overwrite

若仅需路径型Parquet文件（体积更小），可关闭音频字节嵌入功能：
bash
python make_hf_parquet.py
--split-dir train
--path-key auto
--rows-per-shard 10000
--no-embed-audio-bytes
--output-dir train_parquet
--overwrite

## 从Parquet格式解包
该操作可还原音频文件并重新生成与原始路径键值匹配的元数据：
bash
python unpack_hf_parquet.py
--input-dir train_parquet
--output-dir train_unpacked
--overwrite

针对路径型Parquet文件，还需指定原始音频根目录：
bash
python unpack_hf_parquet.py
--input-dir train_parquet
--output-dir train_unpacked
--source-audio-root train
--overwrite