Robust VQA (RVQA) Dataset with Language Prior and Compositional Reasoning Labels

This dataset is designed to advance research in Visual Question Answering (VQA), specifically addressing challenges related to language priors and compositional reasoning. It incorporates question labels categorizing queries based on their susceptibility to either issue, allowing for targeted evaluation of VQA models. The dataset consists of 33,051 training images and 14,165 validation images, along with 571,244 training questions and 245,087 validation questions. Among the training questions, 313,664 focus on compositional reasoning, while 257,580 pertain to language prior. Similarly, the validation questions are categorized into 134,313 for compositional reasoning and 110,774 for language prior. This dataset serves as a benchmarking tool for evaluating models' performance across these two challenges, providing insights into areas that require further improvement. The comprehensive dataset preparation process, including image collection, caption generation, prompt creation, QA pair generation, and quality control, is outlined in the accompanying algorithm. The dataset is designed to be extensible to other image sources and can be a valuable resource for researchers focusing on VQA tasks involving complex reasoning.