team-dentaku/dentaku-llm-as-a-judge

该数据集用于FT-LLM2026数学任务竞赛中团队dentaku的提交系统构建，具体用于训练正误判定预测模型。数据集包含以下列：question（输入数学问题）、code（作为问题解决方案的预期代码）、decision（如果代码被认为是数学问题的有效答案，则为True，否则为False）、rationale（解释最终判断理由的文本）、thinking（推理过程）。在训练正误判定预测模型时，使用了输入提示模板和输出提示模板，并将本数据集的示例填充到这些模板的占位符中进行训练。数据集由2,161,220个实例组成，是通过使用openai/gpt-oss-120b对开发中的解答生成模型输出的代码进行正误判定而创建的。This is the dataset used by Team dentaku to build their submission system for the FT-LLM2026 tuning competition (mathematics task). This repository releases the dataset used for training the correctness judgment prediction model. The column names (keys) and their contents are as follows: question: The input mathematics problem; code: The expected code as a solution to the question; decision: True if the code is considered a valid answer to the given mathematics problem, False otherwise; rationale: Text explaining the reasoning leading to the final decision; thinking: The reasoning process. When training the correctness judgment prediction model, we used the input prompt template for the input and the output prompt template for the output. We trained the model by filling in the placeholders of these templates with the examples from this dataset. The dataset consists of 2,161,220 instances and was created by performing correctness judgment with openai/gpt-oss-120b on the code outputs of an answer generation model under development.