Audio-Turing-Test-Audios

# 📚 Audio Turing Test Audios

> A high‑quality, multidimensional Chinese audio corpus generated from textual transcripts, designed to evaluate the human-likeness and naturalness of Text-to-Speech (TTS) systems—the “Audio Turing Test.”

## About Audio Turing Test (ATT)

ATT is an evaluation framework featuring a standardized human evaluation protocol and an accompanying dataset, addressing the lack of unified evaluation standards in TTS research. To enhance rapid iteration and evaluation, we trained the Auto-ATT model based on Qwen2-Audio-7B, enabling a model-as-a-judge evaluation on the ATT dataset. Full details and related resources are available in the [ATT Collection](https://huggingface.co/collections/meituan/audio-turing-test-682446320368164faeaf38a4).

## Dataset Description

The dataset includes 104 "trap" audio clips for attentiveness checks during evaluations:

* **35 flawed synthetic audio clips:** intentionally synthesized to highlight obvious flaws and unnaturalness.
* **69 authentic human recordings:** genuine human speech, serving as control samples.

## How to Use This Dataset

1. **Evaluate:** Use our [Auto-ATT evaluation model](https://huggingface.co/Meituan/Auto-ATT) to score your own or existing TTS audio clips.
2. **Benchmark:** Compare your evaluation scores against these reference audio samples from top-performing TTS models described in our research paper and these "trap" audio clips.

## Data Format

Audio files are provided in high-quality `.wav` format.

## Citation

If you use this dataset, please cite:

```
@software{Audio-Turing-Test-Audios,
author = {Wang, Xihuai and Zhao, Ziyi and Ren, Siyu and Zhang, Shao and Li, Song and Li, Xiaoyu and Wang, Ziwen and Qiu, Lin and Wan, Guanglu and Cao, Xuezhi and Cai, Xunliang and Zhang, Weinan},
title = {Audio Turing Test: Benchmarking the Human-likeness and Naturalness of Large Language Model-based Text-to-Speech Systems in Chinese},
year = {2025},
url = {https://huggingface.co/datasets/Meituan/Audio-Turing-Test-Audios},
publisher = {huggingface},
}
```

# 📚 音频图灵测试音频库（Audio Turing Test Audios）

> 本数据集为基于文本转录生成的高质量多维度中文音频语料库，旨在评估文本转语音（Text-to-Speech, TTS）系统的类人性与自然性，即「音频图灵测试」。

## 关于音频图灵测试（ATT）

ATT是一套由标准化人工评估协议与配套数据集构成的评估框架，旨在解决当前文本转语音研究领域缺乏统一评估标准的痛点。为加快模型迭代与评估效率，我们基于Qwen2-Audio-7B训练了Auto-ATT模型，可实现以模型作为评判者的ATT数据集评估。完整细节与相关资源可访问[ATT合集](https://huggingface.co/collections/meituan/audio-turing-test-682446320368164faeaf38a4)获取。

## 数据集说明

本数据集包含104段用于评估过程中注意力检查的「陷阱」音频片段：
* **35段存在缺陷的合成音频片段：** 经刻意合成以凸显明显瑕疵与不自然感。
* **69段真实人类录音：** 源自真实人声，作为对照样本。

## 数据集使用方法

1. **评估：** 可使用我们提供的[Auto-ATT评估模型](https://huggingface.co/Meituan/Auto-ATT)对自有或现有文本转语音音频片段进行评分。
2. **基准测试：** 将你的评估得分与本研究论文中提及的顶尖文本转语音模型的参考音频样本，以及上述「陷阱」音频片段进行对比。

## 数据格式

音频文件以高质量`.wav`格式提供。

## 引用声明

若使用本数据集，请引用以下文献：

@software{Audio-Turing-Test-Audios,
author = {Wang, Xihuai and Zhao, Ziyi and Ren, Siyu and Zhang, Shao and Li, Song and Li, Xiaoyu and Wang, Ziwen and Qiu, Lin and Wan, Guanglu and Cao, Xuezhi and Cai, Xunliang and Zhang, Weinan},
title = {Audio Turing Test: Benchmarking the Human-likeness and Naturalness of Large Language Model-based Text-to-Speech Systems in Chinese},
year = {2025},
url = {https://huggingface.co/datasets/Meituan/Audio-Turing-Test-Audios},
publisher = {huggingface},
}