CHATS-Lab/Verbalized-Sampling-Open-Ended-QA

这个数据集展示了如何在保持响应质量的同时，通过使用Verbalized Sampling (VS) 来增加开放式问答中的响应多样性。数据集包含来自各种领域的高级语言模型的开放性问题响应。它评估了响应多样性、质量维持、创意探索和分布覆盖率。数据集结构包括不同的提示方法和对不同语言模型的分割。每个示例包含一个提示、一个包含其概率的响应列表和一个模型标识符。

This dataset demonstrates how Verbalized Sampling (VS) increases response diversity in open-ended question answering while maintaining response quality. It includes diverse responses from various LLMs across different domains and evaluates response diversity, quality maintenance, creative exploration, and distribution coverage. The dataset is structured with different prompting methods and splits for different language models. Each example contains a prompt, a list of responses with their probabilities, and a model identifier.