Speaker-Diarization-Instructions

# Speaker-Diarization-Instructions

Convert diarization dataset from https://huggingface.co/diarizers-community into speech instructions dataset and chunk max to 30 seconds because most of speech encoder use for LLM come from Whisper Encoder.

**We highly recommend to not include AMI test set from both AMI-IHM and AMI-SDM in training set to prevent contamination. This dataset supposely to become a speech diarization benchmark**.

## how to prepare the dataset

```bash
huggingface-cli download \
mesolitica/Speaker-Diarization-Instructions \
--include "*.zip" \
--repo-type "dataset" \
--local-dir './'

unzip 0-0.zip
```

## Acknowledgement

Special thanks to https://www.sns.com.my and Nvidia for 8x H100 node!

# 说话人 diarization（Speaker-Diarization）指令数据集

将来自https://huggingface.co/diarizers-community 的说话人 diarization 数据集转换为语音指令数据集，并将音频片段最大长度限定为30秒，这是因为当前适配大语言模型（LLM）的主流语音编码器均源自Whisper编码器。

**我们强烈建议请勿将AMI-IHM与AMI-SDM中的AMI测试集纳入训练集，以避免数据污染。本数据集旨在成为语音说话人 diarization 基准测试集。**

## 数据集制备流程

bash
huggingface-cli download
mesolitica/Speaker-Diarization-Instructions
--include "*.zip"
--repo-type "dataset"
--local-dir './'

unzip 0-0.zip

## 致谢

特别感谢https://www.sns.com.my 与英伟达（Nvidia）提供的8节点H100算力集群！