VQA-Introspect with Logical Relations

This is a version of the VQA-Introspect dataset by Selvaraju et al., but with added annotations about logical relations for binary QA pairs. Relations have been predicted using a fine-tuned BERT, which was pre-trained for NLI and fine-tuned on a sub-set of VQA-Introspect. In general, entries have the following fields:<br> img_id: This is the image name without extension (images come from COCO)<br> question_id: Question identifier as int<br> sent: String version of the question<br> question_type: Type of question (how it starts)<br> answer_type: Type of answer<br> label: Answers using soft scores (as required by LXMERT)<br> role: Question role (main, sub or unk)<br> Questions with role='sub' also have a field named parent, which indicates the ID of the QA pair it is related to, and a field named rel, which contains the relation to the parent. Total samples:<br> Train: 215862 (sub: 160085, main: 55777)<br> Val: 69668 (sub: 49882, main: 19786) Images must be downloaded separately from the COCO Dataset website. If you use this dataset, please cite: <pre>@inproceedings{tascon2023logical, title={Logical Implications for Visual Question Answering Consistency}, author={Tascon-Morales, Sergio and M{\'a}rquez-Neila, Pablo and Sznitman, Raphael}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={6725--6735}, year={2023} }</pre> as well as the original publication where VQA-Introspect was presented: <pre>@inproceedings{selvaraju2020squinting, title={Squinting at vqa models: Introspecting vqa models with sub-questions}, author={Selvaraju, Ramprasaath R and Tendulkar, Purva and Parikh, Devi and Horvitz, Eric and Ribeiro, Marco Tulio and Nushi, Besmira and Kamar, Ece}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pages={10003--10011}, year={2020} } </pre> Note: We respect the original license terms of the VQA-Introspect dataset and make manifest that the liability warranty described in those terms (§ 4.2 and 4.3) apply to this dataset too.