Sudoku-Bench

该数据集精心挑选了一系列具有挑战性和非常规的数独变体，旨在评估大型语言模型在解决数独问题时所需的创造性多步骤逻辑推理能力。它包含三个子集：挑战100（核心基准）、尼科利100（手工制作的普通数独）以及来自“破解谜题”的ctc（2565个已解决的数独变体）。这些谜题涵盖了4x4、6x6和9x9的不同规模。该数据集的任务是通过解决数独变体来评估大型语言模型中的创造性推理能力。

This dataset carefully curates a collection of challenging and unconventional Sudoku variants, aiming to evaluate the creative multi-step logical reasoning capabilities required for solving Sudoku problems in large language models. It includes three subsets: Challenge100 (core benchmark), Nikoli 100 (handcrafted standard Sudoku), and CTC from "Crack the Puzzle" (2565 solved Sudoku variants). These puzzles cover various grid sizes including 4x4, 6x6, and 9x9. The core task of this dataset is to assess the creative reasoning abilities of large language models by solving these Sudoku variants.

搜集汇总

背景与挑战

背景概述

Sudoku-Bench是一个专门设计用于评估大型语言模型在解决数独变体时创造性多步骤逻辑推理能力的数据集。它包含三个子集：挑战100（核心基准）、尼科利100（手工制作的普通数独）和来自'破解谜题'的ctc（2565个已解决的数独变体），覆盖4x4、6x6和9x9不同规模的谜题，旨在测试模型的推理灵活性和复杂性。