Voices-in-the-Wild-Bench

# Voices-in-the-Wild-Bench Voices-in-the-Wild-Bench is a bilingual benchmark for evaluating robust automatic speech recognition under real-world acoustic conditions. It contains 5,000 speech examples covering Chinese and English, synthetic and real-recorded speech, and eight acoustic scenarios. ## Dataset Composition | Group | Count | |---|---:| | Synthetic speech | 3,500 | | Real-recorded speech | 1,500 | | Chinese | 2,500 | | English | 2,500 | | Total | 5,000 | ## Splits Splits are organized by recording source and acoustic category. Prefixes denote source type: - `real_`: real-recorded speech - `syn_`: synthetic speech Available categories are `noise`, `far_field`, `obstructed`, `distortion`, `recording`, `echo`, `dropout`, and `mixed`. Example split names: ```text real_noise syn_noise real_mixed syn_mixed ``` ## Fields - `audio`: embedded audio feature. - `file_name`: release-relative audio file name. - `audio_path`: release-relative audio path. - `question`: transcription instruction. - `answer`: reference transcription. - `text`: same as `answer`, provided for ASR tooling compatibility. - `subset`: original subset label. - `prediction`: empty placeholder for model outputs. - `name`: sample name. - `index`: sample index. ## Usage ```python from datasets import load_dataset dataset = load_dataset( "zhifeixie/Voices-in-the-Wild-Bench", split="syn_noise", ) sample = dataset[0] audio = sample["audio"] reference = sample["answer"] ``` ## Evaluation We report CER for Chinese samples and WER for English samples. See the project repository and leaderboard for evaluation scripts, model wrappers, and submission instructions: - Repository: https://github.com/xzf-thu/Voices-in-the-Wild-Bench - Leaderboard: https://xzf-thu.github.io/Voices-in-the-Wild-Bench/ ## Citation If you use this benchmark, please cite the associated paper. Citation metadata will be updated after release.