WSC(Winograd Schema Challenge)

Winograd Schema Challenge 的引入既是图灵测试的替代方案，也是对系统进行常识推理能力的测试。 Winograd 模式是一对在一个或两个单词上不同的句子，具有高度模棱两可的代词，在两个句子中的解析方式不同，似乎需要常识知识才能正确解析。这些示例被设计为易于人类解决，但对机器而言却难以解决，原则上需要对文本内容及其描述的情况有深刻的理解。最初的 Winograd Schema Challenge 数据集由 AI 专家手动构建的 100 个 Winograd 模式组成。截至 2020 年，有 285 个示例可用；但是，最后 12 个示例是最近才添加的。为了确保与早期模型的一致性，一些作者通常更愿意只报告前 273 个示例的性能。这些数据集通常分别称为 WSC285 和 WSC273。

The Winograd Schema Challenge was introduced both as an alternative to the Turing Test and a benchmark for evaluating a system's commonsense reasoning capabilities. A Winograd schema is a pair of sentences that differ in one or two words, containing highly ambiguous pronouns whose resolution differs between the two sentences; correctly resolving these pronouns seemingly requires commonsense knowledge. These examples are designed to be readily solvable by humans, yet challenging for machines, as correctly resolving their pronouns in principle requires a profound understanding of the text content and the scenarios they describe. The original Winograd Schema Challenge dataset comprised 100 Winograd schemas manually constructed by AI experts. As of 2020, 285 such examples were available; however, the final 12 examples were only added recently. To ensure consistency with prior models, some authors typically prefer to only report performance on the first 273 examples. These two datasets are commonly referred to as WSC285 and WSC273, respectively.

搜集汇总

背景与挑战

背景概述

WSC（Winograd Schema Challenge）是一个用于测试机器常识推理能力的文本数据集，包含一对仅在少量单词上不同、具有模糊代词的句子，需要常识知识来解析。数据集设计为人类易于解决但对机器具有挑战性，旨在评估系统对文本内容的深层理解，最初有100个示例，后扩展到285个，常使用273个版本以确保一致性。