Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking

Supplementary data for the paper "Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking" at the 15th ACM International WSDM Conference (WSDM 2022).<br>Abstract: Question Answering (QA) has become an often used method to retrieve data as part of chatbots and other natural-language user interfaces. In particular, QA systems of official institutions have high expectations regarding the answers computed by the system, as the provided information might be critical. In this demonstration, we use the official COVID-19 QA system that was developed together with the German Federal government to provide German citizens access to data regarding incident values, number of deaths, etc. To ensure high quality, a component-based approach was used that enables exchanging data between QA components using RDF and validating the functionality of the QA system using SPARQL. Here, we will demonstrate how our solution enables developers of QA systems to use a descriptive approach to validate the quality of their implementation before the system's deployment and also within a live environment.

本数据集为发表于第15届ACM国际网络搜索与数据挖掘大会（ACM International Conference on Web Search and Data Mining，WSDM 2022）的论文《基于组件式微基准测试的德国COVID-19问答系统质量保障》的补充数据集。
摘要：问答系统（Question Answering，QA）现已成为聊天机器人及各类自然语言用户界面中广泛应用的数据检索方法。尤为重要的是，官方机构部署的问答系统对其生成的回答质量有着严苛要求，因为系统所提供的信息往往关乎公众切身利益，具有关键性影响。本演示研究依托与德国联邦政府联合研发的官方COVID-19问答系统，该系统面向德国民众提供新增感染数、死亡人数等相关疫情数据查询服务。为保障系统的高质量运行，本研究采用组件式架构方案：通过资源描述框架（Resource Description Framework，RDF）实现问答系统各组件间的数据交互，并使用SPARQL对问答系统的功能进行验证。在此演示中，我们将展示该解决方案如何帮助问答系统开发者采用描述性方法，在系统部署前乃至实际运行环境中，对其实现方案的质量进行验证。