openbank-uz/youtube_transcriptions

--- language: - uz license: cc-by-nc-4.0 task_categories: - automatic-speech-recognition - text-to-speech tags: - uzbek - speech - audio - tts - asr - youtube - gemini - speaker-clustering pretty_name: Uzbek YouTube Speech Dataset size_categories: - 100K<n<1M --- ## Dataset Description A speech dataset of Uzbek language audio clips sourced from YouTube videos. Audio segments were extracted, separated by speaker using vocal isolation, and transcribed using Google's **Gemini 2.0 Flash** model. Speaker identities were clustered using ECAPA-TDNN embeddings. ### Use Cases - **Automatic Speech Recognition (ASR)** for Uzbek - **Text-to-Speech (TTS)** synthesis for Uzbek - Fine-tuning speech models on Uzbek language data (e.g., Qwen3-TTS) - Speaker-conditioned TTS training ## Code - Scraping and transcription: [Github Repo](https://github.com/Guide-Me-Tech/pdsrt) ## Dataset Structure | Column | Type | Description | |---|---|---| | `audio` | `audio` | Audio column (playable on Hugging Face), 16kHz | | `file` | `string` | Original audio filename | | `gender` | `string` | Speaker gender (`male` / `female`) | | `transcription` | `string` | Uzbek text transcription (generated by Gemini 2.0 Flash) | | `speaker_id` | `string` | Clustered speaker identity (e.g., `spk_131`), 230 unique speakers | | `parsed_time` | `string` | Timestamp when the source video was processed | | `video_title` | `string` | Title of the source YouTube video | | `youtube_video_duration` | `string` | Duration of the source video | | `audio_format` | `string` | Audio encoding format (e.g., `opus32`) | | `speaker_id_in_video` | `string` | Original diarization speaker label within the video (e.g., `01`) | | `speaker_audio_index` | `int` | Index of this audio segment for the speaker within the video | | `prompt_tokens_used` | `int64` | Number of prompt tokens consumed during transcription | | `total_tokens_used` | `int64` | Total tokens consumed during transcription | ### Example ```python { "file": "2025-01-10_20-08-32_umrimiz_asli_qisqa_emas_..._vocalsSPEAKER_01_0.mp3", "gender": "male", "transcription": "Haqiqat qoladi, yo'qolmaydi.", "speaker_id": "spk_131", "parsed_time": "2025-01-10_20-08-32", "video_title": "umrimiz asli qisqa emas kopini bekorga sarflab yuboramiz iqtibos podcast 35", "youtube_video_duration": "31m43s", "audio_format": "opus32", "speaker_id_in_video": "01", "speaker_audio_index": 0, "prompt_tokens_used": 41, "total_tokens_used": 72, "audio": {"path": "...", "array": [...], "sampling_rate": 16000} } ``` ## Data Collection Pipeline 1. **Source**: Public Uzbek-language YouTube videos (podcasts, talks, interviews) 2. **Audio extraction**: Audio tracks extracted from videos and converted to MP3 (Opus 32kbps) 3. **Speaker separation**: Vocals isolated and split by speaker diarization (speaker segments labeled as `SPEAKER_XX`) 4. **Transcription**: Each audio segment transcribed using **Gemini 2.0 Flash** 5. **Gender labeling**: Speaker gender annotated per segment 6. **Speaker clustering**: Global speaker identities assigned across videos (see below) ## Speaker Clustering Speaker identities (`speaker_id`) were assigned across the full dataset using the following method: 1. Speaker Embeddings were created for each audio 2. Embeddings then were clustered to 230 unique spaekers using Mix of two clustering methods ### Embedding Extraction Speaker embeddings were extracted using [SpeechBrain's ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb) model pretrained on VoxCeleb. For code see this notebook: [📒 Google Colab Notebook](https://colab.research.google.com/drive/1YqsvFru-sB_5R85DU1_-qzGxnBaPUk1A?usp=sharing) ### Clustering Method Due to the dataset size (~360K rows), a two-stage clustering approach was used: 1. **Optimal cluster count selection**: Agglomerative Clustering (cosine distance, average linkage) was run on a 10,000-sample subset, testing cluster counts in `range(50, 600, 20)`. The best silhouette score determined the optimal number of clusters. 2. **Full dataset clustering**: MiniBatchKMeans (batch size 4096) was applied to all ~360K samples using the optimal cluster count from step 1. **Result**: 230 unique speakers identified. ```python # Step 1: Find optimal n on subset X_sub = X[:10000] for n in range(50, 600, 20): clustering = AgglomerativeClustering( n_clusters=n, metric="cosine", linkage="average" ) labels = clustering.fit_predict(X_sub) score = silhouette_score(X_sub, labels, metric="cosine") # Step 2: Cluster full dataset with best n clustering = MiniBatchKMeans( n_clusters=best_n, batch_size=4096, random_state=42 ) labels = clustering.fit_predict(X) ``` ### Caveats - Clustering is approximate — some speakers may be split across multiple IDs or merged into one, especially for speakers with similar vocal characteristics. - The original in-video diarization labels (`speaker_id_in_video`) are preserved for cross-referencing. - ECAPA-TDNN was not fine-tuned on Uzbek speech, which may affect clustering quality. ## Limitations - Transcriptions are machine-generated (Gemini 2.0 Flash) and may contain errors, especially for domain-specific vocabulary, names, or dialectal speech. - Audio quality varies depending on the original YouTube source. - Gender labels may not be verified for every segment. - Speaker clustering is unsupervised and approximate — not human-verified. - The dataset is sourced from publicly available YouTube content. If you are a content creator and wish to have your content removed, please open a discussion. ## Citation If you use this dataset, please cite it as: ```bibtex @dataset{openbank-uz/youtube_transcriptions, title={Uzbek YouTube Speech Dataset}, year={2025}, source={YouTube}, transcription_model={Gemini 2.0 Flash}, speaker_clustering={ECAPA-TDNN + MiniBatchKMeans} } ```--- 语言： - 乌兹别克语 许可证：CC BY-NC 4.0 任务类别： - 自动语音识别（Automatic Speech Recognition, ASR） - 文本转语音（Text-to-Speech, TTS） 标签： - 乌兹别克语 - 语音 - 音频 - TTS - ASR - YouTube - Gemini - 说话人聚类 美观名称：乌兹别克语YouTube语音数据集 规模类别： - 10万<样本数<100万 --- ## 数据集描述 本数据集为源自YouTube平台公开视频的乌兹别克语语音片段数据集。我们从视频中提取音频片段，通过人声分离技术按说话人完成分割，并使用谷歌**Gemini 2.0 Flash**模型完成语音转录。随后利用ECAPA-TDNN嵌入向量对说话人身份进行全局聚类。 ### 应用场景 - **乌兹别克语自动语音识别（ASR）** - **乌兹别克语文本转语音（TTS）** 合成 - 基于乌兹别克语数据微调语音模型（例如Qwen3-TTS） - 带说话人条件的TTS模型训练 ## 代码 - 爬取与转录流程：[GitHub仓库](https://github.com/Guide-Me-Tech/pdsrt) ## 数据集结构 | 列名 | 数据类型 | 描述 | |---|---|---| | `audio` | `audio` | 音频列（可在Hugging Face平台直接播放），采样率为16kHz | | `file` | `string` | 原始音频文件名 | | `gender` | `string` | 说话人性别（`male`/`female`，即男性/女性） | | `transcription` | `string` | 乌兹别克语文本转录结果（由Gemini 2.0 Flash生成） | | `speaker_id` | `string` | 聚类后的全局说话人身份（例如`spk_131`），共包含230个唯一说话人 | | `parsed_time` | `string` | 源视频处理完成的时间戳 | | `video_title` | `string` | 源YouTube视频的标题 | | `youtube_video_duration` | `string` | 源YouTube视频的总时长 | | `audio_format` | `string` | 音频编码格式（例如`opus32`） | | `speaker_id_in_video` | `string` | 单视频内原始说话人标注标签（例如`01`） | | `speaker_audio_index` | `int` | 该说话人在当前视频内的音频片段索引 | | `prompt_tokens_used` | `int64` | 转录过程中消耗的提示Token数 | | `total_tokens_used` | `int64` | 转录过程中消耗的总Token数 | ### 示例 python { "file": "2025-01-10_20-08-32_umrimiz_asli_qisqa_emas_..._vocalsSPEAKER_01_0.mp3", "gender": "male", "transcription": "Haqiqat qoladi, yo'qolmaydi.", "speaker_id": "spk_131", "parsed_time": "2025-01-10_20-08-32", "video_title": "umrimiz asli qisqa emas kopini bekorga sarflab yuboramiz iqtibos podcast 35", "youtube_video_duration": "31m43s", "audio_format": "opus32", "speaker_id_in_video": "01", "speaker_audio_index": 0, "prompt_tokens_used": 41, "total_tokens_used": 72, "audio": {"path": "...", "array": [...], "sampling_rate": 16000} } ## 数据采集流程 1. **数据源**：公开的乌兹别克语YouTube视频（涵盖播客、演讲、访谈等内容） 2. **音频提取**：从视频中提取音频轨道，并转换为Opus 32kbps编码的MP3格式 3. **说话人分离**：通过人声分离技术提取人声，基于说话人分割将音频按说话人拆分，片段标注为`SPEAKER_XX` 4. **转录**：使用**Gemini 2.0 Flash**模型对每个音频片段进行转录 5. **性别标注**：为每个音频片段标注对应说话人的性别 6. **说话人聚类**：在全数据集范围内分配全局说话人身份（详见下文） ## 说话人聚类 全数据集范围内的说话人身份（`speaker_id`）通过以下流程分配： 1. 为每个音频片段生成说话人嵌入向量 2. 结合两种聚类方法，将嵌入向量聚类为230个唯一说话人 ### 嵌入向量提取 说话人嵌入向量通过在VoxCeleb数据集上预训练的[SpeechBrain ECAPA-TDNN](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb)模型提取。相关代码可参考该Google Colab笔记本：[📒 Google Colab 笔记本](https://colab.research.google.com/drive/1YqsvFru-sB_5R85DU1_-qzGxnBaPUk1A?usp=sharing) ### 聚类方法 鉴于数据集规模约为36万条数据，我们采用两阶段聚类方案： 1. **最优聚类数选择**：在1万条样本的子集上运行凝聚聚类（余弦距离、平均联接法），测试`range(50, 600, 20)`范围内的聚类数，通过最优轮廓系数确定最佳聚类数。 2. **全数据集聚类**：使用步骤1得到的最优聚类数，对全部约36万条样本应用MiniBatchKMeans（批次大小4096）。 **聚类结果**：共识别出230个唯一说话人。 python # 步骤1：在子集上寻找最优聚类数 X_sub = X[:10000] for n in range(50, 600, 20): clustering = AgglomerativeClustering( n_clusters=n, metric="cosine", linkage="average" ) labels = clustering.fit_predict(X_sub) score = silhouette_score(X_sub, labels, metric="cosine") # 步骤2：使用最优聚类数对全数据集进行聚类 clustering = MiniBatchKMeans( n_clusters=best_n, batch_size=4096, random_state=42 ) labels = clustering.fit_predict(X) ### 注意事项 - 聚类结果为近似结果：部分说话人可能被拆分至多个ID，或被合并为同一ID，尤其针对发声特征相似的说话人。 - 保留单视频内原始说话人标注标签（`speaker_id_in_video`）以供交叉参考。 - ECAPA-TDNN模型未针对乌兹别克语语音进行微调，这可能会影响聚类效果。 ## 局限性 - 转录结果由机器生成（使用Gemini 2.0 Flash），可能存在错误，尤其是针对领域特定词汇、专有名词或方言语音。 - 音频质量因原始YouTube视频源而异。 - 并非每个音频片段的性别标注都经过人工验证。 - 说话人聚类为无监督近似结果，未经过人工验证。 - 本数据集源自公开可获取的YouTube内容。若您为内容创作者并希望移除相关内容，请发起讨论。 ## 引用 若您使用本数据集，请按以下格式引用： bibtex @dataset{openbank-uz/youtube_transcriptions, title={Uzbek YouTube Speech Dataset}, year={2025}, source={YouTube}, transcription_model={Gemini 2.0 Flash}, speaker_clustering={ECAPA-TDNN + MiniBatchKMeans} }