mlcore/phantom-wiki-v050

PhantomWiki是一个用于生成独特、事实一致的文档语料库的框架，其中包含多样化的问答对。与先前的工作不同，PhantomWiki既不是固定数据集，也不基于任何现有数据。相反，每个评估都会按需生成一个新的PhantomWiki实例。PhantomWiki生成一个虚构的宇宙及其相关事实，并在大型语料库中反映这些事实，模仿粉丝维基网站的风格。然后，它生成可调节难度的问答对，包括问答文学中常见的多跳问题类型。

PhantomWiki is a framework for generating unique, factually consistent document corpora with diverse question-answer pairs. Unlike prior work, PhantomWiki is neither a fixed dataset, nor is it based on any existing data. Instead, a new PhantomWiki instance is generated on demand for each evaluation. PhantomWiki generates a fictional universe of characters along with a set of facts. We reflect these facts in a large-scale corpus, mimicking the style of fan-wiki websites. Then we generate question-answer pairs with tunable difficulties, encapsulating the types of multi-hop questions commonly considered in the question-answering (QA) literature.