Finding Relevant User Comments Evaluation collection - FRUCE

FRUCE (Finding Relevant User Comments Evaluation Collection) is an evaluation collection for IR systems created by the Semantic Access and Retrieval (SAR) group of ISL of FORTH. It is specifically designed for evaluating the retrieval of user reviews/comments about hotels. That means, that it contains query-review pairs alongside with their binary relevance, in contrast to the traditional IR evaluation collections, that mainly consist of query-document pairs. The main difference between those two, is the size of the retrieved source, which represents the informative power of the plain text that constitutes the source.