treadon/disinhibition-eval

disinhibition-eval是一个小型评估数据集，用于测量聊天调优语言模型中的对冲与承诺行为。该数据集包含多个分割，旨在测试模型在不同情境下的行为，如对有争议问题的看法、事实正确性、明确的中立性请求、一般连贯性以及在需要对冲的边缘案例中的表现。数据集的设计目的是与对冲/承诺检测器配对使用，以比较基线模型和经过消融的模型。README还提供了模型响应的示例、数据集的模式、预期用例以及相关资源和模型的链接。

disinhibition-eval is a small evaluation dataset for measuring hedging vs. commitment behavior in chat-tuned language models. The dataset includes various splits to test different aspects of model behavior, such as opinions on contentious questions, factual correctness, explicit neutrality requests, general coherence, and edge cases where hedging is appropriate. The dataset is intended to be used with a hedge/commit detector to compare baseline and ablated models. The README also provides examples of model responses, the schema of the dataset, intended use cases, and links to related resources and models.