SalChartQA: Question-driven Saliency on Information Visualisations (Dataset and Reproduction Data)

Understanding the link between visual attention and user’s needs when visually exploring information visualisations is under-explored due to a lack of large and diverse datasets to facilitate these analyses. To fill this gap, we introduce SalChartQA - a novel crowd-sourced dataset that uses the BubbleView interface as a proxy for human gaze and a question-answering (QA) paradigm to induce different information needs in users. SalChartQA contains 74,340 answers to 6,000 questions on 3,000 visualisations. Informed by our analyses demonstrating the tight correlation between the question and visual saliency, we propose the first computational method to predict question-driven saliency on information visualisations. Our method outperforms state-of-the-art saliency models, improving several metrics, such as the correlation coefficient and the Kullback-Leibler divergence. These results show the importance of information needs for shaping attention behaviour and paving the way for new applications, such as task-driven optimisation of visualisations or explainable AI in chart question-answering. The files of this dataset are documented in <a href="https://darus.uni-stuttgart.de/file.xhtml?fileId=276393">README.md</a>.

在视觉探索信息可视化时，视觉注意力与用户需求之间的关联尚未得到充分研究，这主要是因为缺乏足够规模且多样化的数据集来支持此类分析。为填补这一空白，我们提出了SalChartQA——一个全新的众包数据集，它采用BubbleView界面作为人类注视的代理，并通过问答（QA）范式诱发用户的不同信息需求。SalChartQA包含针对3000个可视化图表中6000个问题的74340条回答。基于我们的分析（该分析表明问题与视觉显著性之间存在紧密关联），我们提出了首个计算方法，用于预测信息可视化中由问题驱动的显著性。我们的方法优于当前最先进的显著性模型，在多个指标上均有提升，例如相关系数（correlation coefficient）和Kullback-Leibler散度。这些结果表明信息需求对于塑造注意力行为的重要性，并为新应用铺平了道路，例如可视化的任务驱动优化或图表问答中的可解释AI（Explainable AI）。该数据集的文件记录于<a href="https://darus.uni-stuttgart.de/file.xhtml?fileId=276393">README.md</a>中。