Learning to quantify: LeQua 2022 datasets

# Learning to Quantify The aim of LeQua 2022 (the 1st edition of the CLEF “Learning to Quantify” lab) is to allow the comparative evaluation of methods for “learning to quantify” in textual datasets, i.e., methods for training predictors of the relative frequencies of the classes of interest in sets of unlabelled textual documents. These predictors (called “quantifiers”) will be required to issue predictions for several such sets, some of them characterized by class frequencies radically different from the ones of the training set. ## Links https://lequa2022.github.io/ https://github.com/HLT-ISTI/LeQua2022_scripts ## Tasks T1A: This task is concerned with evaluating binary quantifiers, i.e., quantifiers that must only predict the relative frequencies of a class and its complement. Participants in this task will be provided with documents already converted into vector form; the task is thus suitable for participants who do not wish to engage in generating representations for the textual documents, but want instead to concentrate on optimizing the methods for learning to quantify. T1B: This task is concerned with evaluating single-label multi-class quantifiers, i.e., quantifiers that operate on documents that each belong to exactly one among a set of n>2 classes. Like in Task T1A, participants will be provided with documents already converted in vector form. T2A: Like Task T1A, this task is concerned with evaluating binary quantifiers. Unlike in Task T1A, participants will be provided with the raw text of the documents; the task is thus suitable for participants who also wish to engage in generating suitable representations for the textual documents, or to train end-to-end systems. T2B: Like Task T1B, this task is concerned with evaluating single-label multi-class quantifiers; like in Task T2A, participants will be provided with the raw text of the documents.# 学习量化（Learning to Quantify） LeQua 2022（即CLEF首届“学习量化”实验室）的核心目标是实现文本数据集场景下“学习量化”方法的对比评估，即针对训练预测器的方法展开评估，这类预测器可用于估算未标注文本文档集合中目标类别的相对频率。这类被称为“量化器（quantifiers）”的预测器需对多个此类文档集合做出预测，其中部分集合的类别频率与训练集存在显著差异。 ## 相关链接 https://lequa2022.github.io/ https://github.com/HLT-ISTI/LeQua2022_scripts ## 任务 T1A：本任务用于评估二元量化器，即仅需预测单个类别及其补集相对频率的量化器。参与本任务的选手将获得已转换为向量形式的文档，因此该任务适合不愿参与文本文档表征生成环节，而希望专注于优化学习量化方法的参与者。 T1B：本任务用于评估单标签多分类量化器，即针对每个文档恰好属于n>2个类别集合中某一类的场景开展量化的方法。与任务T1A一致，参与者将获得已转换为向量形式的文档。 T2A：与任务T1A类似，本任务用于评估二元量化器。但与T1A不同的是，参与者将获得文档的原始文本，因此该任务适合希望参与文本文档表征生成，或训练端到端系统的参与者。 T2B：与任务T1B类似，本任务用于评估单标签多分类量化器；与任务T2A一致，参与者将获得文档的原始文本。