PeacefulData/CoVoGER

---
license: cc-by-4.0
language:
- ar
- ca
- cy
- de
- en
- et
- fa
- id
- ja
- lv
- sl
- sv
- ta
- tr
- zh
# Add other specific ISO language codes your dataset supports, e.g., fr, de, zh
multilinguality:
- multilingual
task_categories:
- automatic-speech-recognition
- translation
- text-generation
tags:
- speech-to-text
- generative-error-correction
- n-best-list
- covost2
- common-voice
---

# CoVoGER: A Multilingual Multitask Benchmark for Speech-to-text Generative Error Correction with Large Language Models

## Dataset Description

Large language models (LLMs) can rewrite the N-best hypotheses from a speech-to-text model, often fixing recognition or translation errors that traditional rescoring cannot. Yet research on generative error correction (GER) has been focusing on monolingual automatic speech recognition (ASR), leaving its multilingual and multitask potential underexplored.

We introduce **CoVoGER**, a benchmark for GER that covers both ASR and speech-to-text translation (ST) across 15 languages and 28 language pairs. CoVoGER is constructed by decoding Common Voice 20.0 and CoVoST-2 with Whisper of three model sizes and SeamlessM4T of two model sizes, providing 5-best lists obtained via a mixture of beam search and temperature sampling.

- **Paper:** [CoVoGER: A Multilingual Multitask Benchmark...](https://aclanthology.org/2025.emnlp-main.320/)
- **Repository:** [GitHub - N-Orien/CoVoGER](https://github.com/N-Orien/CoVoGER)
- **Conference:** EMNLP 2025

## Usage and Data Commands

You can easily download and load the dataset using the Hugging Face `datasets` library in Python.

### Terminal/CLI Commands
If you want to download the repository locally via the Hugging Face CLI, run:
```bash
# Ensure you have git-lfs installed
git lfs install
git clone https://huggingface.co/datasets/PeacefulData/CoVoGER
```
Dataset Structure
CoVoGER provides 5-best lists generated by standard ASR and ST models (Whisper and SeamlessM4T). The dataset supports ASR and ST tasks for 15 languages and 28 language pairs.

(Please customize the column names below to exactly match your uploaded Parquet/JSONL files.)

- audio_id: Identifier for the original audio file.

- source_language: Language of the spoken audio.

- target_language: Target language for translation (or the same as the source for ASR).

- n_best_hypotheses: A list of the 5-best transcriptions/translations generated by the base models.

- reference: The ground truth text.

```python
from datasets import load_dataset

# Load the entire dataset
dataset = load_dataset("PeacefulData/CoVoGER")

# Print the first sample of the train split
print(dataset['train'][0])

```

### References

If you use CoVoGER in your research or the work may be relevant, please consider to cite our EMNLP 2025 paper, thank you!

```bib
@inproceedings{yang-etal-2025-covoger,
title = "{C}o{V}o{GER}: A Multilingual Multitask Benchmark for Speech-to-text Generative Error Correction with Large Language Models",
author = "Yang, Zhengdong and
Wan, Zhen and
Li, Sheng and
Yang, Chao-Han Huck and
Chu, Chenhui",
editor = "Christodoulopoulos, Christos and
Chakraborty, Tanmoy and
Rose, Carolyn and
Peng, Violet",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.emnlp-main.320/",
doi = "10.18653/v1/2025.emnlp-main.320",
pages = "6302--6314",
ISBN = "979-8-89176-332-6",
abstract = "Large language models (LLMs) can rewrite the N-best hypotheses from a speech-to-text model, often fixing recognition or translation errors that traditional rescoring cannot. Yet research on generative error correction (GER) has been focusing on monolingual automatic speech recognition (ASR), leaving its multilingual and multitask potential underexplored. We introduce CoVoGER, a benchmark for GER that covers both ASR and speech-to-text translation (ST) across 15 languages and 28 language pairs. CoVoGER is constructed by decoding Common Voice 20.0 and CoVoST-2 with Whisper of three model sizes and SeamlessM4T of two model sizes, providing 5-best lists obtained via a mixture of beam search and temperature sampling. We evaluated various instruction-tuned LLMs, including commercial models in zero-shot mode and open-sourced models with LoRA fine-tuning, and found that the mixture decoding strategy yields the best GER performance in most settings. CoVoGER will be released to promote research on reliable language-universal speech-to-text GER. The code and data for the benchmark are available at https://github.com/N-Orien/CoVoGER."
}

许可证：CC BY 4.0
支持语言：
- 阿拉伯语（ar）
- 加泰罗尼亚语（ca）
- 威尔士语（cy）
- 德语（de）
- 英语（en）
- 爱沙尼亚语（et）
- 波斯语（fa）
- 印尼语（id）
- 日语（ja）
- 拉脱维亚语（lv）
- 斯洛文尼亚语（sl）
- 瑞典语（sv）
- 泰米尔语（ta）
- 土耳其语（tr）
- 中文（zh）
# 添加您的数据集支持的其他ISO语言代码，例如fr、de、zh
多语言属性：
- 多语言
任务类别：
- 自动语音识别
- 机器翻译
- 文本生成
标签：
- 语音转文本
- 生成式错误修正
- N最优列表
- CoVoST-2
- Common Voice

# CoVoGER：面向大语言模型（Large Language Model，LLM）语音转文本生成式错误修正的多语言多任务基准数据集

## 数据集概述
大语言模型（LLM）可重写语音转文本模型输出的N最优假设，通常能修复传统重打分无法修正的识别或翻译错误。但当前生成式错误修正（Generative Error Correction，GER）的研究多聚焦于单语言自动语音识别（Automatic Speech Recognition，ASR），使其多语言与多任务潜力未得到充分挖掘。

我们提出**CoVoGER**，一款面向GER的基准数据集，覆盖15种语言与28种语言对的自动语音识别与语音转文本翻译（Speech-to-Text Translation，ST）任务。CoVoGER通过使用3种模型尺寸的Whisper与2种模型尺寸的SeamlessM4T对Common Voice 20.0与CoVoST-2进行解码构建，提供通过束搜索与温度采样混合策略生成的5最优列表。

- **论文**：[CoVoGER: A Multilingual Multitask Benchmark...](https://aclanthology.org/2025.emnlp-main.320/)
- **代码仓库**：[GitHub - N-Orien/CoVoGER](https://github.com/N-Orien/CoVoGER)
- **收录会议**：EMNLP 2025

## 使用方法与数据指令
您可通过Python的Hugging Face `datasets`库轻松下载并加载该数据集。

### 终端/命令行指令
若您希望通过Hugging Face CLI本地下载该仓库，请执行：
bash
# 确保已安装git-lfs
git lfs install
git clone https://huggingface.co/datasets/PeacefulData/CoVoGER

### 数据集结构
CoVoGER提供由标准ASR与ST模型（Whisper与SeamlessM4T）生成的5最优列表。该数据集支持15种语言的ASR与ST任务，涵盖28种语言对。

（请将下方的列名调整为与您上传的Parquet/JSONL文件完全匹配）
- `audio_id`：原始音频文件的标识符
- `source_language`：口语音频的源语言
- `target_language`：翻译的目标语言（对于ASR任务则与源语言一致）
- `n_best_hypotheses`：基础模型生成的5个最优转录/翻译结果列表
- `reference`：基准真值文本

python
from datasets import load_dataset

# 加载完整数据集
dataset = load_dataset("PeacefulData/CoVoGER")

# 打印训练集拆分的第一个样本
print(dataset['train'][0])

### 引用说明
若您在研究中使用CoVoGER或相关工作，请考虑引用我们的EMNLP 2025论文，感谢您的支持！

bib
@inproceedings{yang-etal-2025-covoger,
title = "{C}o{V}o{GER}: 面向大语言模型语音转文本生成式错误修正的多语言多任务基准数据集",
author = "Yang, Zhengdong and Wan, Zhen and Li, Sheng and Yang, Chao-Han Huck and Chu, Chenhui",
editor = "Christodoulopoulos, Christos and Chakraborty, Tanmoy and Rose, Carolyn and Peng, Violet",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2025.emnlp-main.320/",
doi = "10.18653/v1/2025.emnlp-main.320",
pages = "6302--6314",
ISBN = "979-8-89176-332-6",
abstract = "大语言模型（LLM）可重写语音转文本模型输出的N最优假设，通常能修复传统重打分无法修正的识别或翻译错误。但当前生成式错误修正（GER）的研究多聚焦于单语言自动语音识别（ASR），使其多语言与多任务潜力未得到充分探索。我们提出CoVoGER，一款面向GER的基准数据集，覆盖15种语言与28种语言对的ASR与语音转文本翻译（ST）任务。CoVoGER通过使用3种模型尺寸的Whisper与2种模型尺寸的SeamlessM4T对Common Voice 20.0与CoVoST-2进行解码构建，提供通过束搜索与温度采样混合策略生成的5最优列表。我们评估了多种指令微调大语言模型，包括零样本模式下的商用模型与基于LoRA微调的开源模型，发现混合解码策略在多数设置下可取得最优的GER性能。CoVoGER将开源以推动面向可靠的跨语言通用语音转文本GER研究。该基准的代码与数据可在https://github.com/N-Orien/CoVoGER获取。"
}