Kid-FRIEND: An Evaluation Dataset for Web Search Kid-Friendliness

This kid-FRIEND dataset allows to evaluate the child-friendliness web search. We compare three general and three child-oriented web search engines based on a new German evaluation corpus (including DeepL-based translations to English) of 50 queries spanning personal, political, educational, and entertainment information needs of children. For each query, we annotate the search engines’ top-10 results with respect to relevance and potential harm: a child-friendly result should be relevant but harmless. Version 1 of this dataset contained some duplicated and some inconsistent relevance judgments in the qrels. For the updated version 2, we removed those duplicated entries and re-judged entries that had inconsistent relevance judgments.