sarvamai/audiollm-evals

---
dataset_info:
features:
- name: question
dtype: string
- name: lang
dtype: string
- name: audio
dtype: audio
splits:
- name: train
num_bytes: 218348705
num_examples: 980
download_size: 212931635
dataset_size: 218348705
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
pretty_name: Indic AudioLLM Evals
license: apache-2.0
---

This evaluation set contains ~100 questions in both text and audio format in Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu. We use this dataset internally at Sarvam to evaluate the performance of our audio models. We open-source this data to enable the research community to replicate the results mentioned in our [Shuka blog](https://www.sarvam.ai/blogs/shuka-v1).

By deisgn, the questions are sometimes vague, and the audio has noise and other inconsistencies, to measure the robustness of models.