EQueR Evaluation Package

The EQueR Evaluation Package was produced within the French national project EQueR (Evaluation campaign for Question-Answering systems), as part of the Technolangue programme funded by the French Ministry of Research and New Technologies (MRNT). The EQueR project enabled to carry out a campaign for the evaluation of Question-Answering systems in French.This package includes the material that was used for the EQueR evaluation campaign. It includes resources, protocols, scoring tools, results of the campaign, etc., that were used or produced during the campaign. The aim of these evaluation packages is to enable external players to evaluate their own system and compare their results with those obtained during the campaign itself. The campaign is distributed over two actions: 1)Generic task: it consists in evaluating the performances of question-answering systems on a collection of heterogeneous texts.2)Specialised task: it consists in evaluating the performances of question-answering systems on a collection of texts from the medical domain.The EQueR evaluation package contains the following data and tools: 1)Two text collections:-General corpus: about 1.5 Gb of data consisting of news articles of several years from Le Monde and Le Monde Diplomatique, press releases and information reports from the French Senate dealing with various subjects.-Medical corpus: about 140 Mb of data mainly consisting of scientific articles and guidelines for good medical practice, selected by the CISMeF (Catalogue et Index des Sites Médicaux Francophones) from the University Hospital Centre of Rouen.1)Two corpora of questions :-500 questions for the generic task and 200 questions for the specialised task.-For each question in the two corpora, the first 100 identifiers are provided (from Pertimm’s search engine).2)Two Pertimm’ sub-corpora, created from the document identifiers and returned by the search engine.3)The whole results provided by the participants.4)A help software for the evaluation of results within the evaluation of question-answering systems (with detailed documentation).A description of the project is available at the following address:http://www.technolangue.net/article.php3?id_article=195 (in French language)

EQueR评估包系法国国家项目EQueR（问答系统评估活动）的成果，该项目由法国研究与新技术部（MRNT）资助的Technolangue计划支持。EQueR项目旨在开展针对法语问答系统的评估活动。本包内含EQueR评估活动所使用的全部材料，包括资源、协议、评分工具、活动结果等。这些评估包的目的是为了使外部参与者能够评估自身系统，并将评估结果与活动期间所获得的结果进行比较。活动分为两个部分：1）通用任务：评估问答系统在异构文本集合上的性能。2）专业任务：评估问答系统在医学领域文本集合上的性能。EQueR评估包包含以下数据和工具：1）两个文本集合：-通用语料库：约1.5Gb的数据，包括来自《世界报》和《世界报外交版》的多年新闻文章、法国参议院的新闻稿和信息报告。-医学语料库：约140Mb的数据，主要包含来自鲁昂大学医院的科学文章和良好医疗实践指南，由CISMeF（法语医学网站目录与索引）精选。-两个问题语料库：用于通用任务的500个问题和用于专业任务的200个问题。对于两个语料库中的每个问题，提供了前100个标识符（来自Pertimm搜索引擎）。2）两个由文档标识符和搜索引擎返回的Pertimm子语料库。3）参与者提供的全部结果。4）一套用于问答系统评估结果评估的帮助软件（附带详细文档）。项目描述可访问以下地址：http://www.technolangue.net/article.php3?id_article=195（法语）