sarvamai/indivibe

indivibe是一个用于评估Sarvam模型在22种印度预定语言上的能力的基准数据集。该数据集旨在反映印度当前语言使用的实际情况，因此每种语言都提供两种脚本风格：原生脚本（正式书面使用）和罗马化拉丁脚本（常见于消息传递和在线交流的口语使用）。数据集分为四个领域：通用聊天、STEM、数学和编码。基准源自110个英文源提示：50个通用聊天提示，以及STEM、数学和编码各20个提示。每个提示被翻译成所有22种印度预定语言，并以原生和罗马化脚本提供，每种语言产生220个提示，总共4,840个提示（22种语言 × 2种脚本 × 110个提示）。通用聊天领域涵盖创意写作、文化知识、日常建议、情感支持和轻度推理。STEM、数学和编码领域旨在更直接地评估多语言推理和技术问题解决能力。由于所有语言都使用相同基础源提示的翻译，模型在每种语言和脚本上都在语义等效任务上接受评估，从而实现公平的跨语言比较，同时允许按领域独立分析对话和推理性能。

indivibe is a benchmark dataset for evaluating the capabilities of Sarvam models across 22 scheduled Indian languages. The dataset is designed to reflect how language is actually used in India today, evaluating each language in two script styles: native script (formal written usage) and romanized Latin script (colloquial usage commonly seen in messaging and online communication). It is organized into four domains: general chat, STEM, mathematics, and coding. The benchmark originates from 110 English source prompts: 50 general chat and 20 each for STEM, mathematics, and coding. Each prompt is translated into all 22 scheduled Indian languages and provided in both native and romanized script, yielding 220 prompts per language and 4,840 prompts in total (22 languages × 2 scripts × 110 prompts). The general chat domain covers creative writing, cultural knowledge, everyday advice, emotional support, and light reasoning. The STEM, mathematics, and coding domains are designed to more directly evaluate multilingual reasoning and technical problem-solving. Because all languages use translations of the same underlying source prompts, models are evaluated on semantically equivalent tasks across every language and script, enabling fair cross-lingual comparison while allowing independent analysis of conversational and reasoning performance by domain.