RoleEval

RoleEval是一个双语角色评估基准，旨在评估角色知识的记忆、利用和推理能力。它包括全球范围内认可的角色（RoleEval-Global）和中国流行的角色（RoleEval-Chinese），包含6000个中英文平行多项选择题，关注300个有影响力的人物和虚构角色，涵盖名人、动漫、漫画、电影、电视剧、游戏和小说等多个领域。这些问题覆盖了基本知识和多跳推理能力，旨在系统地探究角色的个人信息、关系、能力和经历等各个方面。RoleEval is a bilingual role evaluation benchmark aimed at assessing the memory, utilization, and reasoning abilities of role-related knowledge. It encompasses two subsets: RoleEval-Global for globally recognized roles and RoleEval-Chinese for popular roles in China. The benchmark contains 6,000 parallel multiple-choice questions in both Chinese and English, centering on 300 influential public figures and fictional characters across diverse domains including celebrities, anime, comics, films, television series, video games, and novels. These questions cover both basic knowledge and multi-hop reasoning skills, with the goal of systematically investigating various aspects of the characters, such as their personal information, interpersonal relationships, capabilities, and life experiences.