Creative Invention Benchmark (CrIB)

Creative Invention Benchmark (CrIB) 是一个包含2000个问题的基准数据集，旨在评估计算创造力的特定方面。数据集由交互计算学院创建，涵盖绘画、外星语言、图像拼贴、叙事和甜点食谱五个领域。每个问题要求代理从给定的知识库中概括和发明新概念，以达到未知目标的解决方案。数据集创建过程中，采用了多种生成策略和初始基准。CrIB的应用领域广泛，从产品发明到药物发现，旨在模拟人类通过反思现有知识来发明激进解决方案的过程，并期望人工智能代理也能做到这一点。

Creative Invention Benchmark (CrIB) is a benchmark dataset containing 2000 questions, designed to evaluate specific aspects of computational creativity. Developed by the School of Interactive Computing, the dataset covers five domains: painting, alien languages, image collaging, narrative, and dessert recipes. For each question, an agent is required to generalize and invent new concepts from a provided knowledge base to produce solutions for unknown goals. Multiple generation strategies and initial benchmarks were adopted during the dataset construction process. CrIB has wide-ranging applications spanning from product invention to drug discovery, aiming to simulate the human process of inventing radical solutions by reflecting on existing knowledge, with the expectation that AI agents can achieve this capability.