vakyansh/truthfulqa_indic

--- license: apache-2.0 task_categories: - text-generation language: - hi - pa - te - ta - kn size_categories: - 1K<n<10K dataset_info: - config_name: hi features: - name: type dtype: string - name: category dtype: string - name: question dtype: string - name: best_answer dtype: string - name: correct_answers sequence: string - name: incorrect_answers sequence: string - name: source dtype: string splits: - name: hi num_examples: 817 - name: pa num_examples: 817 - name: te num_examples: 817 - name: ta num_examples: 817 - name: kn num_examples: 817 --- [Original Repository](https://github.com/sylinrl/TruthfulQA) ## Tasks (from original repository) ### Generation (main task): Task: Given a question, generate a 1-2 sentence answer. Objective: The primary objective is overall truthfulness, expressed as the percentage of the model's answers that are true. Since this can be gamed with a model that responds "I have no comment" to every question, the secondary objective is the percentage of the model's answers that are informative. ### Future Work: 1. Validate individual data files with Language Experts 2. Add evaluation scripts 3. Benchmark GPT3.5, GPT-4, LLaMa-2, OpenHathi 4. Add evaluation metrics

原始信息汇总

数据集概述

许可证

• Apache 2.0

任务类别

• 文本生成

语言

• 印地语 (hi)
• 旁遮普语 (pa)
• 泰卢固语 (te)
• 泰米尔语 (ta)
• 卡纳达语 (kn)

数据集大小

• 1K<n<10K

数据集信息

• 配置名称: hi
• 特征:
  • type: 字符串
  • category: 字符串
  • question: 字符串
  • best_answer: 字符串
  • correct_answers: 字符串序列
  • incorrect_answers: 字符串序列
  • source: 字符串
• 分割:
  • hi: 817个样本
  • pa: 817个样本
  • te: 817个样本
  • ta: 817个样本
  • kn: 817个样本