ChatRAG-Hi

## Dataset Description:
The ChatRAG-Hi (Hindi ChatRAG Bench) dataset is based on the English version of the ChatRAG Bench, which comprises the following ten datasets: Doc2Dial, QuAC, QReCC, INSCIT, HybriDialogue, DoQA, and ConvFinQA. The dataset was translated using GCP, and approximately 500 samples were filtered from each of these sets based on backtranslation accuracy to eliminate poor translations.<br>
The evaluation steps are described [here](https://huggingface.co/datasets/nvidia/ChatRAG-Hi/blob/main/evaluation/README.md).

## Dataset Owner:
NVIDIA Corporation

## Dataset Creation Date:
April 2025

## License/Terms of Use:
This dataset is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0).

## Intended Usage:
This dataset is used to evaluate a large language model’s (LLM) conversational QA capability over documents or retrieved context in the Hindi Language.

## Dataset Characterization
Data Collection Method<br>
* Synthetic <br>

Labeling Method<br>
* Synthetic <br>

## Dataset Format
Text

## Dataset Quantification
474MB of prompt-response pairs, comprising 5948 individual samples.

## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility, and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Citing

If you find our work helpful, please consider citing our paper:
```
@article{kamath2025benchmarking,
title={Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis},
author={Kamath, Anusha and Singla, Kanishk and Paul, Rakesh and Joshi, Raviraj and Vaidya, Utkarsh and Chauhan, Sanjay Singh and Wartikar, Niranjan},
journal={arXiv preprint arXiv:2508.19831},
year={2025}
}
```

数据集描述：
ChatRAG-Hi（印地语ChatRAG基准测试集，Hindi ChatRAG Bench）以英文版本的ChatRAG基准测试集为基础构建，该基准测试集包含以下十个数据集：Doc2Dial、QuAC、QReCC、INSCIT、HybriDialogue、DoQA以及ConvFinQA。本数据集通过谷歌云平台（Google Cloud Platform，GCP）完成翻译，并基于回译准确率从每个数据集集中筛选约500个样本，以剔除质量不佳的翻译结果。评估步骤详见[此处](https://huggingface.co/datasets/nvidia/ChatRAG-Hi/blob/main/evaluation/README.md)。

数据集所有者：英伟达公司（NVIDIA Corporation）

数据集创建日期：2025年4月

许可与使用条款：本数据集采用知识共享署名-相同方式共享4.0国际许可协议（Creative Commons Attribution-ShareAlike 4.0 International License，CC BY-SA 4.0）进行许可。

预期用途：本数据集用于评估大语言模型（Large Language Model，LLM）在印地语语境下，基于文档或检索得到的上下文完成对话式问答的能力。

数据集特征：
数据收集方式：合成生成
标注方式：合成生成

数据集格式：文本格式

数据集规模：包含5948条独立样本，总数据量为474MB的提示-响应对。

伦理考量：
英伟达认为可信人工智能是一项共同责任，我们已制定相关政策与实践规范，以支持各类人工智能应用的开发。开发者在遵循本服务条款的前提下下载或使用本数据集时，应与其内部模型团队协作，确保该模型符合相关行业与应用场景的要求，并应对潜在的产品滥用问题。
请通过[此处](https://www.nvidia.com/en-us/support/submit-security-vulnerability/)报告安全漏洞或英伟达人工智能相关问题。

引用说明：
若您认为本研究对您有所帮助，请引用以下论文：

@article{kamath2025benchmarking,
title={Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis},
author={Kamath, Anusha and Singla, Kanishk and Paul, Rakesh and Joshi, Raviraj and Vaidya, Utkarsh and Chauhan, Sanjay Singh and Wartikar, Niranjan},
journal={arXiv preprint arXiv:2508.19831},
year={2025}
}