small-overlapping-speech-bench
收藏资源简介：
Small Overlapping Speech Bench是一个小型、完全可复现的多语言重叠语音基准测试数据集，旨在对自动语音识别和说话人日志模型进行“鸡尾酒会”式的压力测试，评估模型在多人同时说话场景下的性能。数据集包含100个音频片段，每个片段时长约15至26秒，为单声道16 kHz MP3格式。每个片段模拟三人同时说话的场景，三人分别使用三种不同的欧洲语言：德语、法语、西班牙语、意大利语、荷兰语和葡萄牙语。数据集提供精确的真实标注，包括每个片段的唯一标识符、总时长、说话人数量、语言列表，以及针对每个说话人的详细信息，如说话人ID、所用语言、语言代码、源说话人ID、在混合音频中的精确起止时间戳和转录文本，这些时间戳信息也使数据集可作为说话人日志任务的真实标注。数据来源于Multilingual LibriSpeech数据集，通过可复现的流程构建：从源数据中筛选特定长度和说话人多样性的单说话人语音片段，进行RMS响度归一化后，以交错重叠的时间安排进行混合，并求和限幅输出最终片段。评估采用内容词召回率指标，以解决重叠多语言语音中词序未定义的问题。数据集适用于多语言自动语音识别、重叠语音处理、说话人日志和“鸡尾酒会问题”相关研究。需要注意的是，语音源自有声书朗读，因此重叠是合成的而非自然对话；荷兰语片段较少；数据集规模较小，主要用于快速、低成本的模型压力测试，且混合条件理想化，如等响度、无混响和噪声。数据集采用CC-BY-4.0许可。
The Small Overlapping Speech Bench is a small, fully reproducible multilingual overlapping speech benchmark dataset designed as a "cocktail party" stress test for automatic speech recognition (ASR) and speaker diarization models, to evaluate their performance in scenarios with multiple simultaneous speakers. This dataset contains 100 audio clips, each with a duration of approximately 15 to 26 seconds, and is provided in mono-channel 16 kHz MP3 format. Each clip simulates a scenario with three simultaneous speakers, each using one of six distinct European languages: German, French, Spanish, Italian, Dutch, and Portuguese. It provides precise ground-truth annotations for every clip, including unique clip identifiers, total duration, number of speakers, language list, and detailed per-speaker information such as speaker ID, language used, language code, source speaker ID, exact start and end timestamps in the mixed audio, and transcript text. These timestamp annotations also enable the dataset to serve as ground truth for the speaker diarization task. The dataset is derived from the Multilingual LibriSpeech dataset and constructed via a reproducible workflow: single-speaker speech clips with specific duration and speaker diversity are first filtered from the source data, then normalized via RMS loudness normalization, mixed with staggered overlapping timing, and finally output via summation and clipping. Evaluation uses the content word recall metric to address the issue of undefined word order in overlapping multilingual speech. This dataset is suitable for research on multilingual automatic speech recognition, overlapping speech processing, speaker diarization, and related studies on the "cocktail party problem". It should be noted that the speech sources are derived from audiobook readings, so the overlapping is synthetic rather than natural conversation; there are relatively few Dutch clips; the dataset is small in scale, primarily intended for fast, low-cost stress testing of models, and the mixing conditions are idealized, such as equal loudness, no reverb, and no noise. The dataset is licensed under CC-BY-4.0.
数据集概述
数据集名称：Small Overlapping Speech Bench
许可证：CC-BY-4.0
任务类别：自动语音识别 (ASR)
语言：德语、法语、西班牙语、意大利语、荷兰语、葡萄牙语
数据集大小：少于 1000 条样本
标签：重叠语音、多语言、说话人分离、ASR 基准、鸡尾酒会问题、语音
数据集内容
该数据集是一个小型的、完全可复现的多语言重叠语音基准测试集。包含 100 个音频片段，每个片段中有 三个人同时用三种不同的欧洲语言说话，并提供了每个说话人的真实时间戳、语言和转录文本。
- 音频片段：100 个，每个包含 3 个同步说话人（使用 3 种语言），时长约 15-26 秒，单声道，16 kHz MP3 格式。
- 语言：德语、法语、西班牙语、意大利语、荷兰语、葡萄牙语。
- 数据来源：基于
facebook/multilingual_librispeech(CC-BY-4.0) 构建。 - 包含内容：真实标注、评估脚本及 6 个 ASR 模型的结果。
数据文件结构
data/ clip_000.mp3 … clip_099.mp3 # 100 个重叠语音片段 metadata.jsonl # 每个片段的真实标注 (HF AudioFolder 格式) ground_truth.jsonl # 包含 "audio_file" 路径字段的真实标注 scripts/ # 重建和评估所需的脚本 metrics.json # 6 个 ASR 模型的基准测试得分 RESULTS.md / benchmark_report.html# 可读的结果报告
真实标注格式 (每个片段示例)：
json { "clip_id": "clip_000", "duration": 26.18, "num_speakers": 3, "languages": ["italian", "portuguese", "french"], "speakers": [ {"id": "S1", "language": "italian", "code": "it", "src_speaker_id": 280, "start_time": 0.0, "end_time": 12.98, "transcript": "chiamando buon vulcano …"}, {"id": "S2", "language": "portuguese", "code": "pt", "src_speaker_id": 12287, "start_time": 6.717, "end_time": 17.097,"transcript": "constituíam uma destas …"}, {"id": "S3", "language": "french", "code": "fr", "src_speaker_id": 1406, "start_time": 12.59, "end_time": 26.18, "transcript": "sécria dabord le génie …"} ] }
使用方法
使用 datasets 库 (自动解码音频)：
python from datasets import load_dataset ds = load_dataset("laion/small-overlapping-speech-bench", split="test") ex = ds[0] ex["audio"] # {array: np.ndarray (16 kHz mono), sampling_rate: 16000} ex["speakers"] # list of 3 dicts: language, code, start_time, end_time, transcript
不使用 datasets 库 (直接访问文件)：
python from huggingface_hub import snapshot_download import json, librosa, os
root = snapshot_download("laion/small-overlapping-speech-bench", repo_type="dataset") gt = [json.loads(l) for l in open(os.path.join(root, "ground_truth.jsonl"))] for clip in gt: wav, sr = librosa.load(os.path.join(root, clip["audio_file"]), sr=16000, mono=True) for spk in clip["speakers"]: print(clip["clip_id"], spk["code"], spk["start_time"], spk["transcript"][:40])
构建方法
- 来源池：从
facebook/multilingual_librispeech的测试集中获取 6 种语言的单说话人语音片段。 - 混合：为每个片段选择 3 种不同语言，将每个说话人的音频 RMS 归一化至同等响度，并以交错重叠的起始时间放置，叠加后导出为 16 kHz 单声道 MP3 文件。
- 随机性：所有随机过程均有种子控制，可逐比特重建。
评估指标
由于重叠多语言语音导致词错误率 (WER) 不适用，因此采用 内容词召回率 (Content-Word Recall)：
- 内容词：指长度 ≥ 3 个字母的单词。
- 评分方式：对于每个真实标注的语音段，计算其内容词出现在模型转录文本中的比例，并报告总体、按语言、以及最好/最差说话人的召回率。
- 捕获数 (capt/3)：表示 3 个说话人中，有多少个被成功捕获（召回率 ≥ 30%）。
- 说话人计数：对于说话人分离模型，报告其预测的说话人数与真实值（3）的对比。
部分模型测试结果 (内容词召回率)
| 模型 | 总体 | 最好/3 | 最差/3 | capt/3 |
|---|---|---|---|---|
| Nemotron 3.5 ASR — 1.12 s 块 | 38.8 | 65.4 | 10.4 | 1.86 |
| Nemotron 3.5 ASR — 80 ms 块 | 33.8 | 60.4 | 7.9 | 1.62 |
| Qwen3-ASR-1.7B | 25.5 | 62.1 | 1.1 | 1.03 |
| VibeVoice-ASR | 25.3 | 43.8 | 5.4 | 1.27 |
| Chorus-v1 | 16.3 | 37.7 | 1.2 | 0.55 |
| Parakeet TDT v3 | 11.6 | 30.6 | 0.2 | 0.48 |
- 核心结论：目前没有模型能完全转录所有三个说话人；Nemotron 3.5 转录能力最强；Sortformer 说话人计数能力最优。
已知局限
- 朗读语音：来源于有声读物，非自然对话，重叠为人工合成。
- 荷兰语样本少：MLS 荷兰语测试集说话人较少。
- 规模小：仅 100 个片段，适合快速基准测试，非大规模评估集。
- 混合方式：等响度混合，未模拟真实场景中的音量变化、混响和噪声。
许可与引用
- 许可：CC-BY-4.0，继承自源语料库 Multilingual LibriSpeech。
- 引用：请引用 Vinay Pratap 等人的论文《MLS: A Large-Scale Multilingual Dataset for Speech Research》。
