DL-HARD (Deep Learning Hard)

Deep Learning Hard (DL-HARD) 是一个带注释的数据集，旨在更有效地评估复杂主题的神经排序模型。它建立在 TREC 深度学习 (DL) 问题的基础上，并使用来自领先网络搜索引擎的查询意图类别、答案类型、维基化实体、主题类别和结果类型元数据进行了广泛注释。
DL-HARD 包含来自官方 2019/2020 评估基准的 50 个查询，其中一半是新的独立评估。总体而言，DL-HARD 是一种新资源，通过专注于具有挑战性和复杂的查询来促进对神经排序方法的研究。

Deep Learning Hard (DL-HARD) is an annotated dataset designed to more effectively evaluate neural ranking models on complex topics. It is built upon TREC Deep Learning (DL) queries, and has been extensively annotated with metadata including query intent categories, answer types, wikified entities, topic categories, and result type metadata sourced from leading web search engines.
DL-HARD includes 50 queries from the official 2019/2020 evaluation benchmarks, half of which serve as new independent evaluation sets. Overall, DL-HARD is a novel resource that promotes research on neural ranking methods by focusing on challenging and complex queries.

搜集汇总

背景与挑战

背景概述

DL-HARD是一个带注释的数据集，旨在更有效地评估复杂主题的神经排序模型。它基于TREC深度学习问题，包含50个查询，其中一半为新的独立评估，专注于挑战性查询以促进神经排序方法的研究。