AIML-TUDA/SLR-Homes

SLR-Homes是一个可扩展逻辑推理基准的分布外领域变体，专注于房屋分类任务。该数据集用于归纳逻辑编程任务，要求模型根据房屋的房间组成和属性，区分现代和传统房屋。数据集包含500个示例，均匀分布在10个难度级别（每级50个示例）。每个示例提供自然语言提示，包含正负示例和背景事实，模型需要归纳出能够完美区分两类房屋的Prolog规则。数据集使用与SLR-Bench相同的符号验证器进行评估，仅将正负谓词改为modern和traditional。房屋中的每个房间通过8个谓词描述：has_room（房间标识）、room_num（房间号）、has_wall_color（墙面颜色）、has_roof_type（屋顶类型）、has_garden（花园类型）、has_garage（车库类型）、has_window（窗户类型）和window_num（窗户数量）。数据集列包括：id（唯一标识）、prompt（自然语言指令）、ground-truth rule（潜在Prolog规则）、validation program（可执行验证程序）、level（难度级别1-10）、num_houses（房屋总数）、num_rooms（每房屋房间数）、rule_length（规则体长度）、kappa_positive（正例数）、kappa_negative（负例数）。难度级别参数随级别增加而增加，涉及房屋数量、房间数量和规则长度。

SLR-Homes is an out-of-distribution domain variant of SLR-Bench, applied to the domain of classifying Houses (composed of Rooms with various properties) as modern or traditional. It is generated by the SLR framework and supports the same automated symbolic evaluation as SLR-Bench. The dataset contains 500 examples in a single test split, balanced across 10 difficulty levels (50 examples per level). The task format is identical to SLR-Bench: each example provides a natural-language prompt with positive/negative examples plus background facts, and the model must induce a Prolog rule that separates the two classes. Evaluation uses the same symbolic judge as SLR-Bench, with the only change being the positive_predicate/negative_predicate pair (modern/traditional instead of eastbound/westbound). Each Room is described by eight predicates: has_room, room_num, has_wall_color, has_roof_type, has_garden, has_garage, has_window, and window_num. Dataset columns include: id, prompt, ground-truth rule, validation program, level, num_houses, num_rooms, rule_length, kappa_positive, and kappa_negative. Difficulty levels range from 1 to 10, with increasing parameters for num_houses, num_rooms, and rule_length.