HDJin/RegexPSPACE

RegexPSPACE是一个为了评估大型语言模型（LLM）在PSPACE完整正则表达式问题上的推理能力而设计的新基准。该数据集通过双指数空间探索和有效的过滤过程，从超过一百万个初始实例中筛选出1685个具有挑战性的问题。数据集分为测试集和少量样本集，每个实例包含一个具有挑战性的正则表达式问题及其不同任务的真实解决方案。RegexPSPACE is a new benchmark designed to evaluate the reasoning capabilities of Large Language Models (LLMs) on PSPACE-complete regex problems. The dataset was constructed through double-exponential space exploration and a sound filtering process, curating 1,685 challenging problems from over a million initial instances. It is divided into a test split and a fewshot split, with each instance containing a challenging regex problem and its associated ground-truth solutions for different tasks.