LLM4Code/SATBench

SATBench是一个包含2,100个谜题的数据集，旨在通过从SAT公式自动生成谜题来评估大型语言模型在逻辑推理方面的表现。每个谜题都包含维度结构、变量数量、子句数量、可读的CNF公式、公式是否可满足、背景语境、变量到现实世界意义的映射、自然语言约束以及一个最终问题。数据集以JSONL格式存储。

SATBench is a dataset consisting of 2,100 puzzles designed to benchmark the logical reasoning capabilities of large language models by generating puzzles from SAT formulas automatically. Each puzzle includes dimensional structure, number of variables, number of clauses, readable CNF formula, satisfiability of the formula, background context, mapping from variables to real-world meanings, natural language constraints, and a final question. The dataset is stored in JSONL format.