The Impact of Evidence Order on LLMs in Fact Verification

This repository contains the data to replicate the results of the paper title: "The Impact of Evidence Order on LLMs in Fact Verification".

Contents

The repository includes the following files:

filler.txt: Contains a dummy passage about llamas sourced from Wikipedia that was used as unrelated text for testing evidence placement within different context lengths (as referenced in Section 3.3 of the paper)

hover_dataset.pqt: Contains the HOVER dataset that includes evidence from multiple Wikipedia articles per claim, creating multi-hop verification scenarios that mimic real-world fact-checking complexities

feverous_dev.pqt: Contains the FEVEROUS dataset which expands upon the FEVER dataset by incorporating structured data and more complex claims requiring verification

Related Code Repository

The code used to analyze this data and reproduce the results presented in the paper can be found in a separate repository:

https://anonymous.4open.science/r/Needle_In_Factchecking/README.md