GLUE

The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems. GLUE consists of:

A benchmark of nine sentence- or sentence-pair language understanding tasks built on established existing datasets and selected to cover a diverse range of dataset sizes, text genres, and degrees of difficulty,

A diagnostic dataset designed to evaluate and analyze model performance with respect to a wide range of linguistic phenomena found in natural language, and

A public leaderboard for tracking performance on the benchmark and a dashboard for visualizing the performance of models on the diagnostic set.

The format of the GLUE benchmark is model-agnostic, so any system capable of processing sentence and sentence pairs and producing corresponding predictions is eligible to participate. The benchmark tasks are selected so as to favor models that share information across tasks using parameter sharing or other transfer learning techniques. The ultimate goal of GLUE is to drive research in the development of general and robust natural language understanding systems.

Dataset webpage: https://gluebenchmark.com/

通用语言理解评估基准测试集（General Language Understanding Evaluation，GLUE）是一套用于训练、评估与分析自然语言理解系统的资源集合。GLUE包含以下三部分：
1. 基于现有成熟数据集构建的9项单句或句对级语言理解任务基准测试集，所选任务涵盖了多样化的数据集规模、文本体裁与难度梯度；
2. 一套专为评估与分析模型在自然语言中各类语言现象上表现而设计的诊断数据集；
3. 一个用于追踪基准测试集表现的公开排行榜，以及一个用于可视化模型在诊断数据集上表现的交互式仪表盘。

该基准测试集的格式与模型无关，因此任何能够处理单句及句对并生成对应预测结果的系统均可参与。所选基准任务倾向于支持通过参数共享或其他迁移学习技术实现跨任务信息共享的模型。GLUE的最终目标是推动通用且鲁棒的自然语言理解系统研发领域的研究进展。

数据集官网：https://gluebenchmark.com/