s-nlp/hotpotqa-dev-llama3.1-8b-instruct-temp0.9-samples99-logprobs

--- license: mit task_categories: - text-generation - question-answering language: - en pretty_name: HotpotQA Llama 3.1 8B Self-Consistency Samples with Post-hoc Logprobs tags: - hotpotqa - self-consistency - logprobs - llama-3.1 - reasc --- # HotpotQA Llama 3.1 8B Self-Consistency Samples with Post-hoc Logprobs This dataset contains the HotpotQA self-consistency generations for Llama 3.1 8B with post-hoc token-level log-probabilities added for each sampled completion. Source generations: - `memyprokotow/hotpotqa_dev_Llama3.1-8b-instruct_temp0.9_samples99` Scoring model: - `unsloth/Llama-3.1-8B-Instruct` Main file: - `predictions.parquet` Auxiliary file: - `merge_metadata.json` Important columns: - `question` - `answer` - `all_completions` - `all_logprobs` - `final_answer` - `num_scored_completions` Notes: - `all_logprobs[i]` contains token-level log-probabilities for `all_completions[i]`. - This artifact was produced by post-hoc scoring stored completions rather than by regenerating the samples.许可证：MIT许可证 任务类别： - 文本生成 - 问答 语言： - 英语 数据集展示名：HotpotQA Llama 3.1 8B 带事后对数概率（logprobs）的自一致性（self-consistency）样本 标签： - HotpotQA - 自一致性（self-consistency） - 对数概率（logprobs） - Llama-3.1 - reasc # 带事后对数概率的Llama 3.1 8B自一致性HotpotQA样本数据集 本数据集包含针对Llama 3.1 8B模型的HotpotQA自一致性生成结果，并为每个采样完成结果添加了事后Token级对数概率。 源生成数据集： - `memyprokotow/hotpotqa_dev_Llama3.1-8b-instruct_temp0.9_samples99` 评分模型： - `unsloth/Llama-3.1-8B-Instruct` 主文件： - `predictions.parquet` 辅助文件： - `merge_metadata.json` 重要字段： - `question`：问题文本 - `answer`：参考标准答案 - `all_completions`：所有采样完成结果 - `all_logprobs`：各采样结果对应的对数概率 - `final_answer`：模型最终输出答案 - `num_scored_completions`：参与评分的采样结果数量 备注： - `all_logprobs[i]` 包含`all_completions[i]`对应的Token级对数概率。 - 该数据集产物是通过对已存储的采样完成结果进行事后评分得到的，而非重新生成样本。