Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking

Supplementary data for the paper "Quality Assurance of a German COVID-19 Question Answering Systems using Component-based Microbenchmarking" at the 15th ACM International WSDM Conference (WSDM 2022).<br>Abstract: Question Answering (QA) has become an often used method to retrieve data as part of chatbots and other natural-language user interfaces. In particular, QA systems of official institutions have high expectations regarding the answers computed by the system, as the provided information might be critical. In this demonstration, we use the official COVID-19 QA system that was developed together with the German Federal government to provide German citizens access to data regarding incident values, number of deaths, etc. To ensure high quality, a component-based approach was used that enables exchanging data between QA components using RDF and validating the functionality of the QA system using SPARQL. Here, we will demonstrate how our solution enables developers of QA systems to use a descriptive approach to validate the quality of their implementation before the system's deployment and also within a live environment.

本数据集为发表于第15届ACM国际Web搜索与数据挖掘大会（WSDM 2022）的论文《基于组件式微基准测试的德语COVID-19问答系统质量保障》的补充数据。<br>摘要：问答系统（Question Answering，QA）已成为聊天机器人及其他自然语言用户界面中用于检索数据的常用方法。官方机构部署的问答系统对系统生成的答案有着极高要求，因其所提供的信息可能关乎重大公共利益。在本次演示研究中，我们采用了与德国联邦政府联合开发的官方COVID-19问答系统，该系统面向德国民众提供新增感染数、死亡数等相关数据的查询服务。为保障系统高质量运行，我们采用了组件式架构方案：该方案允许问答系统各组件间通过资源描述框架（Resource Description Framework，RDF）交换数据，并使用SPARQL查询语言（SPARQL）验证问答系统的功能完整性。本次演示将展示，我们的解决方案可帮助问答系统开发者采用描述性方法，在系统部署前乃至实际运行环境中验证其实现方案的质量。