Data for: CLAIRE: A combinatorial visual analytics system for information retrieval evaluation

We considered the following standard and shared collec- tions, each track using 50 different topics:

• TREC Adhoc tracks T07 and T08: they focus on a news search task and adopt a corpus of about 528K news documents.
• TREC Web tracks T09 and T10: focus on a Web search task and adopt a corpus of 1.7M Web pages.
• TREC Terabyte tracks T14 and T15: focus on a Web search task and adopt a corpus of 125M Web pages.
We considered three main components of an IR system: stop list, stemmer, and IR model. We selected a set of alternative implementations of each component and, by using the Ter- rier v.4.02 open source system, we created a run for each system defined by combining the available components in all possible ways. The selected components are:

• Stop list: nostop, indri, lucene, snowball,
smart, terrier;
• Stemmer: nolug, weakPorter, porter,
snowballPorter, krovetz, lovins;
• Model: bb2, bm25, dfiz, dfree, dirichletlm, dlh, dph, hiemstralm, ifb2, inb2, inl2, inexpb2, jskls, lemurtfidf, lgd, pl2, tfidf.

Overall, these components define a 6 × 6 × 17 factorial design with a GoP consisting of 612 system runs. They represent nearly all the state-of-the-art components which constitute the common denominator almost always present
in any IR system for English retrieval and thus they are a good account of what can be found in many different operational settings.