mteb/MemBench

MemBench是一个基于MTEB（大规模文本嵌入基准）的数据集，专门用于对话记忆检索任务。它测量模型在偏好、情感、推荐、更新和多跳记忆线索方面的检索能力。数据集来源于LMEB（长时记忆嵌入基准），包含多个子任务配置，如聚合性、比较性、情感、知识更新、多跳、多会话助理、后处理、偏好、单跳和单会话助理等。每个配置都包含语料库（corpus）、查询（queries）、相关性评分（qrels）和排名靠前的结果（top_ranked）。数据集仅包含测试分割，适用于评估文本嵌入模型在社交和口语领域的性能。MemBench is an MTEB (Massive Text Embedding Benchmark) dataset designed for dialogue-memory retrieval tasks, measuring retrieval for preference, emotion, recommendation, update, and multi-hop memory cues. It is derived from the LMEB (Long-horizon Memory Embedding Benchmark) and includes multiple configurations such as aggregative, comparative, emotion, knowledge updating, multi-hop, multi-session assistant, post-processing, preference, single-hop, and single-session assistant. Each configuration consists of a corpus, queries, qrels (relevance judgments), and top-ranked results. The dataset only contains a test split and is intended for evaluating text embedding models in social and spoken domains.