U4RASD/Muradif

Muradif（مُرادِف，意为“同义词”）是一个基于同义词的基准测试，用于直接评估嵌入质量，无需额外的微调。每个数据行是一个包含上下文的三个词组合：模型需要将带有锚词（anchor_word）的上下文（context）嵌入更接近带有同义词（syn_word）的上下文，而不是带有无关词（irrelevant_word）的上下文。该基准测试包含38,554个这样的三元组。此数据集在第64届计算语言学协会年会（ACL 2026）上发布。

Muradif (مُرادِف, "synonym") is a synonym-based benchmark that directly assesses embedding quality with no additional fine-tuning. Each row is a triplet with a context: a model should embed `context` with `anchor_word` closer to `context` with `syn_word` (a true synonym) than to `context` with `irrelevant_word` (an unrelated word). There are 38,554 triplets in this benchmark. This dataset was introduced at the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026).