sudoku-bench-nikoli

<h1 align="center"> <b>Sudoku-Bench: nikoli_100</b><br> </h1> <p align="center"> 🐙 <a href="https://github.com/SakanaAI/Sudoku-Bench">[Sudoku-Bench GitHub]</a><br> 📝 <a href="https://arxiv.org/abs/2505.16135">[Technical Report]</a> </p> **The Sudoku-Bench project has been discontinued.** The `nikoli_100` puzzle collection remains available for download here. ## Dataset description The `nikoli_100` dataset contains 100 beautiful handmade standard Sudoku puzzles designed by [Nikoli](https://www.nikoli.co.jp/en/), the Japanese puzzle company that popularized Sudoku in the 1980s. The collection was curated in partnership with Nikoli as part of Sudoku-Bench, a benchmark for evaluating the reasoning capabilities of large language models. [Algorithmically generated Sudoku puzzles](https://www.kaggle.com/datasets/rohanrao/sudoku) tend to find only puzzles of a certain type, namely whose solution path is similarly algorithmic. Human setters are more capable of creating puzzles that require deeper reasoning and creativity in the solve: see [this video](https://www.youtube.com/watch?v=mlLq8qaTLBo) for an example. ## Quickstart ```python import datasets nikoli_100 = datasets.load_dataset("SakanaAI/sudoku-bench-nikoli", split="test") ``` ## Puzzle fields Each puzzle contains the fields: - `puzzle_id`: Identifier for the puzzle - `sudokupad_url`: Link to play the puzzle on [SudokuPad](https://sudokupad.app) - `author`: Creator of the puzzle (`Nikoli` for all puzzles in this collection) - `title`: Name of the puzzle - `rules`: The puzzle rules (standard 9×9 Sudoku rules for all puzzles in this collection) - `initial_board`: 81-character string representation of the starting grid in row-major order, with empty cells shown as `.` - `solution`: 81-character string representation of the completed grid - `rows`: Number of rows in the puzzle (always `9`) - `cols`: Number of columns in the puzzle (always `9`) - `visual_elements`: JSON-encoded string describing visual components for Sudoku variants. All puzzles in this collection are standard Sudoku, so this field is always the empty list `[]` - `encoded_puzzle`: A compressed representation of the puzzle using SudokuPad's encoding scheme. `https://sudokupad.app/{encoded_puzzle}` loads the puzzle in SudokuPad without a call to the SudokuPad puzzle database ## Example puzzle The first puzzle in the dataset: - `puzzle_id`: `'nhk2212m'` - `sudokupad_url`: `'https://sudokupad.app/u3hluueg8w'` - `author`: `'Nikoli'` - `title`: `'Nikoli nhk2212m'` - `rules`: `'"Normal 9x9 sudoku rules apply."'` - `initial_board`: `'7..6....8...9..5....6.4..3.23...8.....4...6.....7...91.2..3.4....1..9...8....1..'` - `solution`: `'74561392831298754698654213723916875417439568265872439152783641946125987389347126'` - `rows`: `9` - `cols`: `9` - `visual_elements`: `'[]'` - `encoded_puzzle`: `'sclN4SwJgXA5AdgFgawExIIxILZQDQYKYAuAhmEcRMEQK4FwD2ATtAHIgJ...'` (truncated) ## License and attribution This dataset is released under the [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) license. Please credit [Nikoli](https://www.nikoli.co.jp/en/) when using these puzzles. We are grateful to Nikoli for curating the `nikoli_100` collection. ## Citation ```bibtex @misc{seely2025sudoku, title={Sudoku-Bench: Evaluating creative reasoning with Sudoku variants}, author={Jeffrey Seely and Yuki Imajuku and Tianyu Zhao and Edoardo Cetin and Llion Jones}, year={2025}, eprint={2505.16135}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2505.16135}, } ```