Bangla Information Retrieval Test Collection | Revisiting Anwesha

There are several IR test collections available in English (e.g. http://ir.dcs.gla.ac.uk/resources/test_collections/). Unfortunately, no Gold standard dataset existed for Bangla IR until recently (https://zenodo.org/record/6583149). Our work expands the existing Gold standard dataset by creating 100 query document relevance pairs across a new test collection of 1000 documents. The corpus contains news articles from Ebela, Zee News and Anandabazar Patrika, Vikaspedia and various Bangla travel blogs. The definition of the complexity level of a query is described below:

Complexity Level 1: The query contains exact words, phrases or sentence from the document.

Complexity Level 2: The query is not present as it is in the document. There is a slight deviation.

Complexity Level 3: The query is a generalised phrase capturing the overall story or the document’s theme.

Complexity Level 4: It is a general query not related to any specific document.