WiC-TSV (Words-in-Context: Target Sense Verification)

我们提出了 WiC-TSV，一种新的词义消歧多域评估基准。更具体地说，我们介绍了一个用于在上下文中对单词进行目标意义验证的框架，该框架将其在公式中的独特性作为一个二进制分类任务，因此独立于外部意义清单和各个领域的覆盖范围。这使得数据集高度灵活，可用于评估域内和跨域的各种模型和系统。根据提供给模型的输入信号，WiC-TSV 提供三种不同的评估设置。

We introduce WiC-TSV, a novel multi-domain evaluation benchmark for word sense disambiguation. More specifically, we present a framework for target word sense verification in context, which frames the task as a binary classification problem based on the contextual uniqueness of the target word, thus being independent of external sense inventories and domain-specific coverage requirements. This renders the dataset highly flexible for evaluating diverse models and systems across both in-domain and cross-domain scenarios. Depending on the input signals provided to the model, WiC-TSV offers three distinct evaluation settings.