MInference/SCBench

SCBench（SharedContextBench）是一个综合基准，用于评估高效长上下文方法，特别关注KV缓存的生命周期（生成、压缩、检索和加载）。该数据集包含12个不同的任务，测试了四个关键的长上下文能力：字符串检索、语义检索、全局信息处理和多任务处理。数据集支持两种共享上下文模式：多轮模式和多请求模式。SCBench是第一个覆盖单轮、多轮和多请求场景的长上下文基准，并涉及KV缓存重用技术，从而提供了对高效长上下文方法在完整KV缓存生命周期中的更全面分析。

SCBench (SharedContextBench) is a comprehensive benchmark to evaluate efficient long-context methods in a KV cache-centric perspective. It covers 12 diverse tasks that test four key long-context capabilities: string retrieval, semantic retrieval, global information processing, and multi-tasking. The dataset includes various configurations such as multi_turn_choice_eng, multi_turn_kv, etc., each with different features and training data. The dataset description mentions two shared context modes: multi-turn mode and multi-request mode. Additionally, it discusses comparisons with previous long-context benchmarks and key findings and results.