MTUCI/openstt_balalaika

---
license: cc-by-nc-4.0
task_categories:
- text-to-speech
- automatic-speech-recognition
language:
- ru
pretty_name: OpenSTT annotate by Balalaika
---

# OpenSTT Annotated by Balalaika

**A curated Russian speech dataset for advanced speech generative tasks.**

## Overview

**OpenSTT Annotated by Balalaika** is a high-quality Russian speech corpus, meticulously filtered and annotated by the **lab260 team at MTUCI** with the latest version of our pipeline, **BALALAIKA**.

- **Language:** Russian only
- **Genres:** Podcasts, public speech, YouTube, audiobooks, phone calls, TTS, and more
- **Source:** OpenSTT ([GitHub link](https://github.com/snakers4/open_stt?tab=readme-ov-file))
- **License:** CC BY-NC 4.0 (same as original OpenSTT)
- **Total Duration After Filtering:** 431.43 hours (from over 20,108 hours raw)
- **Format:** Parquet files with split-wise annotation

***

## Usage

**Primary Use Cases:**
- Text-to-Speech (TTS) generation
- Automatic Speech Recognition (ASR)
- Analysis of accent, stress, and prosody
- Russian speech technology research

### 1. Download the dataset

### 2. Extract the files

```basg
for archive in *.tar.gz; do
dir="${archive%.tar.gz}"
mkdir -p "$dir"
tar -xzvf "$archive" -C "$dir"
rm "$archive"
done
```

### 3. Load data in PyTorch

```python
from pathlib import Path
import pandas as pd
from torch.utils.data import Dataset
import torchaudio

class ParquetConcatDataset(Dataset):
def __init__(self, parquet_dir, audio_root, parse_fn=None):
self.parquet_dir = Path(parquet_dir)
self.audio_root = Path(audio_root)

parquet_files = list(self.parquet_dir.glob("*.parquet"))
dfs = [pd.read_parquet(f) for f in parquet_files]
self.df = pd.concat(dfs, ignore_index=True)

def __len__(self):
return len(self.df)

def __getitem__(self, idx):
row = self.df.iloc[idx]
audio_path = self.audio_root / row["filepath"]
waveform, sample_rate = torchaudio.load(audio_path)
return {
"audio_path": str(audio_path),
"waveform": waveform,
"sample_rate": sample_rate,
"nisqa_mos": row["mos_pred"],
"nisqa_noi": row["noi_pred"],
"nisqa_dis": row["dis_pred"],
"nisqa_col": row["col_pred"],
"nisqa_loud": row["loud_pred"],
"nisqa_model": row["model"],
"is_single_speaker": bool(row["is_single_speaker"]),
"accented_text": row["accent"],
"asr_text": row["rover"],
"punctuated_text": row["punct"],
"phonemes": row["phonemes"]
}

# Example usage
ds = ParquetConcatDataset(
PATH_TO_PARQUETS_DIR,
PATH_TO_AUDIO_ROOT
)
```

`PATH_TO_PARQUETS_DIR`: Path to the folder containing all .parquet files with metadata and annotations for the dataset.

`PATH_TO_AUDIO_ROOT`: Path to the root directory containing all audio subfolders and files referenced by filepath columns in the metadata.
***

## Data Processing & Annotation

Our pipeline applies **rigorous filtering and enrichment** steps:

1. **Removed speech segments** shorter than 3 seconds
2. **Filtered segments** with [NISQA](https://github.com/gabrielmittag/NISQA/tree/master/nisqa) MOS < 4.0 for quality assurance
3. **Excluded segments with multiple speakers** (via [pyannotate diarization](https://huggingface.co/pyannote/speaker-diarization-community-1))
4. **Filtered out speech with music background** (custom music detector)
5. **Revised transcriptions:** Crowd-sourced with multiple ASRs, fused via ROVER ([T-one](https://github.com/voicekit-team/T-one/tree/main), [GigaAMv2-rnnt, GigaAMv2-ctc, GigaAMv2-ctc-lm](https://github.com/salute-developers/GigaAM), [vosk](https://huggingface.co/alphacep/vosk-model-ru))
6. **Punctuation added** using [RuPunct](https://huggingface.co/RUPunct/RUPunct_big)
7. **Stress marks added** via [RuAccent](https://github.com/Den4ikAI/ruaccent)
8. **IPA phonemization** performed with our own neural model

All **annotation fields** are handled and provided separately for transparency and flexibility.

***

## Data Structure

- **Annotation storage:** Parquet files
- **Speech storage:** .tar.gz files with speech segments in .opus
- **Splitting:** Follows OpenSTT splits
- **Annotations:** Each sample includes separate fields for:
- **Filepath**
- **Quality metrics: MOS, NOI, DIS, COL, LOUD**
- **Model for quality assesment**
- **Transcript with stresses and pucntuation**
- **Transcript after ROVER**
- **Transcript with punctuation**
- **IPA transcription**
- **Speaker diarization flag**

***

## How to Cite

Please cite the following paper if you use this dataset in research:
```
@misc{borodin2025datacentricframeworkaddressingphonetic,
title={A Data-Centric Framework for Addressing Phonetic and Prosodic Challenges in Russian Speech Generative Models},
author={Kirill Borodin and Nikita Vasiliev and Vasiliy Kudryavtsev and Maxim Maslov and Mikhail Gorodnichev and Oleg Rogov and Grach Mkrtchian},
year={2025},
eprint={2507.13563},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2507.13563},
}
```

***

## Contact

- Telegram: [@korallll_ai](https://t.me/korallll_ai)
- Email: [[e-mail]](mailto:[e-mail])

***

## Links

- [Balalaka annotation pipeline](https://github.com/mtuciru/balalaika/tree/main/src)
- [Other datasets annotated by BALALAIKA](https://huggingface.co/collections/MTUCI/balalaika-dataset)
- [Custom models' inference implementaton](https://huggingface.co/collections/MTUCI/balalaika-models)
- [Paper (arXiv)](https://arxiv.org/pdf/2507.13563)
- [OpenSTT repository](https://github.com/snakers4/open_stt?tab=readme-ov-file)
- [NISQA](https://github.com/gabrielmittag/NISQA/tree/master/nisqa)
- [pyannotate diarization](https://huggingface.co/pyannote/speaker-diarization-community-1)
- [T-one](https://github.com/voicekit-team/T-one/tree/main)
- [GigaAMv2-rnnt, GigaAMv2-ctc, GigaAMv2-ctc-lm](https://github.com/salute-developers/GigaAM)
- [vosk](https://huggingface.co/alphacep/vosk-model-ru)
- [RuPunct](https://huggingface.co/RUPunct/RUPunct_big)
- [RuAccent](https://github.com/Den4ikAI/ruaccent)

***

## License

Distributed under **CC BY-NC 4.0**, matching original OpenSTT terms.

***

license: CC BY-NC 4.0
task_categories:
- 文本转语音（Text-to-Speech, TTS）
- 自动语音识别（Automatic Speech Recognition, ASR）
language:
- ru
pretty_name: Balalaika标注版OpenSTT数据集
---

# Balalaika标注版OpenSTT数据集

**一款经精心整理的俄语语音数据集，适用于高端语音生成类任务。**

## 概述

**Balalaika标注版OpenSTT数据集是一款高品质俄语语音语料库，由莫斯科国立通信与信息技术大学（MTUCI）的lab260团队依托最新版本的BALALAIKA标注流水线精心筛选并完成标注。**

- **语言：** 仅包含俄语语音数据
- **体裁：** 播客、公开演讲、YouTube内容、有声书、电话通话、文本转语音（TTS）内容等
- **来源：** OpenSTT数据集（[GitHub链接](https://github.com/snakers4/open_stt?tab=readme-ov-file)）
- **许可协议：** CC BY-NC 4.0（与原始OpenSTT数据集的许可条款一致）
- **筛选后总时长：** 431.43小时（原始数据总时长超过20108小时）
- **存储格式：** 采用Parquet文件存储分块标注信息

***

## 使用指南

**核心应用场景：**
- 文本转语音（TTS）生成
- 自动语音识别（ASR）
- 口音、重音及韵律特征分析
- 俄语语音技术相关研究

### 1. 下载数据集

### 2. 解压文件

bash
for archive in *.tar.gz; do
dir="${archive%.tar.gz}"
mkdir -p "$dir"
tar -xzvf "$archive" -C "$dir"
rm "$archive"
done

### 3. 在PyTorch中加载数据

python
from pathlib import Path
import pandas as pd
from torch.utils.data import Dataset
import torchaudio

class ParquetConcatDataset(Dataset):
def __init__(self, parquet_dir, audio_root, parse_fn=None):
self.parquet_dir = Path(parquet_dir)
self.audio_root = Path(audio_root)

parquet_files = list(self.parquet_dir.glob("*.parquet"))
dfs = [pd.read_parquet(f) for f in parquet_files]
self.df = pd.concat(dfs, ignore_index=True)

def __len__(self):
return len(self.df)

def __getitem__(self, idx):
row = self.df.iloc[idx]
audio_path = self.audio_root / row["filepath"]
waveform, sample_rate = torchaudio.load(audio_path)
return {
"audio_path": str(audio_path),
"waveform": waveform,
"sample_rate": sample_rate,
"nisqa_mos": row["mos_pred"],
"nisqa_noi": row["noi_pred"],
"nisqa_dis": row["dis_pred"],
"nisqa_col": row["col_pred"],
"nisqa_loud": row["loud_pred"],
"nisqa_model": row["model"],
"is_single_speaker": bool(row["is_single_speaker"]),
"accented_text": row["accent"],
"asr_text": row["rover"],
"punctuated_text": row["punct"],
"phonemes": row["phonemes"]
}

# Example usage
ds = ParquetConcatDataset(
PATH_TO_PARQUETS_DIR,
PATH_TO_AUDIO_ROOT
)

`PATH_TO_PARQUETS_DIR`: 指向存储数据集所有元数据与标注信息的.parquet文件所在文件夹的路径。

`PATH_TO_AUDIO_ROOT`: 指向存储所有音频子文件夹与文件的根目录，该目录下的文件路径需与元数据中filepath列的内容匹配。

***

## 数据处理与标注流程

本数据集依托的标注流水线包含以下严格的筛选与增强步骤：
1. **移除短语音片段：** 移除时长低于3秒的语音片段
2. **质量筛选：** 过滤NISQA语音质量评估工具预测的平均主观意见分（MOS）低于4.0的片段，以保障数据整体质量
3. **单说话人筛选：** 排除存在多位说话人的片段（通过[pyannotate说话人 diarization](https://huggingface.co/pyannote/speaker-diarization-community-1)工具实现）
4. **背景音乐过滤：** 移除带有背景音乐的语音片段（基于自定义音乐检测器）
5. **转录文本融合：** 采用多种自动语音识别模型生成转录结果，通过ROVER算法融合得到最终转录文本，所用模型包括[T-one](https://github.com/voicekit-team/T-one/tree/main)、[GigaAMv2-rnnt、GigaAMv2-ctc、GigaAMv2-ctc-lm](https://github.com/salute-developers/GigaAM)以及[vosk](https://huggingface.co/alphacep/vosk-model-ru)
6. **标点符号添加：** 借助[RuPunct](https://huggingface.co/RUPunct/RUPunct_big)模型为转录文本添加标点符号
7. **重音标记添加：** 通过[RuAccent](https://github.com/Den4ikAI/ruaccent)工具为俄语文本添加标准重音标记
8. **音素标注：** 依托自研神经网络模型完成国际音标（International Phonetic Alphabet, IPA）音素序列标注

所有标注字段均单独提供，以保障数据集的透明性与使用灵活性。

***

## 数据存储结构

- **标注数据存储：** 采用Parquet文件格式
- **语音数据存储：** 采用.tar.gz压缩包存储.opus格式的语音片段
- **数据集划分：** 完全沿用OpenSTT数据集的原始划分规则
- **标注字段说明：** 每个样本包含以下独立字段：
- **文件路径（Filepath）：** 对应音频文件的相对路径
- **质量评估指标：** MOS（平均主观意见分）、NOI（噪声评分）、DIS（失真评分）、COL（色彩失真评分）、LOUD（响度评分）
- **质量评估模型：** 用于生成上述质量指标的评估模型
- **带重音与标点的转录文本**
- **ROVER融合后的ASR转录文本**
- **仅带标点的转录文本**
- **国际音标（IPA）音素序列**
- **单说话人标记：** 标识该片段是否仅包含单个说话人的语音

***

## 引用方式

若您在研究工作中使用本数据集，请引用以下论文：

@misc{borodin2025datacentricframeworkaddressingphonetic,
title={A Data-Centric Framework for Addressing Phonetic and Prosodic Challenges in Russian Speech Generative Models},
author={Kirill Borodin and Nikita Vasiliev and Vasiliy Kudryavtsev and Maxim Maslov and Mikhail Gorodnichev and Oleg Rogov and Grach Mkrtchian},
year={2025},
eprint={2507.13563},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2507.13563},
}

***

## 联系方式

- 电报（Telegram）：[@korallll_ai](https://t.me/korallll_ai)
- 电子邮箱：[[e-mail]](mailto:[e-mail])

***

## 相关链接

- [Balalaika标注流水线](https://github.com/mtuciru/balalaika/tree/main/src)
- [其他由BALALAIKA标注的数据集](https://huggingface.co/collections/MTUCI/balalaika-dataset)
- [自研模型推理实现方案](https://huggingface.co/collections/MTUCI/balalaika-models)
- [论文（arXiv）](https://arxiv.org/pdf/2507.13563)
- [OpenSTT开源仓库](https://github.com/snakers4/open_stt?tab=readme-ov-file)
- [NISQA语音质量评估工具](https://github.com/gabrielmittag/NISQA/tree/master/nisqa)
- [pyannotate说话人 diarization工具](https://huggingface.co/pyannote/speaker-diarization-community-1)
- [T-one ASR模型](https://github.com/voicekit-team/T-one/tree/main)
- [GigaAM系列ASR模型](https://github.com/salute-developers/GigaAM)
- [vosk ASR工具](https://huggingface.co/alphacep/vosk-model-ru)
- [RuPunct标点恢复模型](https://huggingface.co/RUPunct/RUPunct_big)
- [RuAccent重音标注工具](https://github.com/Den4ikAI/ruaccent)

***

## 许可协议

本数据集采用**CC BY-NC 4.0**许可协议，与原始OpenSTT数据集的许可条款完全一致。

***