The Impact of Evidence Order on LLMs in Fact Verification

This repository contains the data to replicate the results of the paper title: "The Impact of Evidence Order on LLMs in Fact Verification". Contents The repository includes the following files: filler.txt: Contains a dummy passage about llamas sourced from Wikipedia that was used as unrelated text for testing evidence placement within different context lengths (as referenced in Section 3.3 of the paper) hover_dataset.pqt: Contains the HOVER dataset that includes evidence from multiple Wikipedia articles per claim, creating multi-hop verification scenarios that mimic real-world fact-checking complexities feverous_dev.pqt: Contains the FEVEROUS dataset which expands upon the FEVER dataset by incorporating structured data and more complex claims requiring verification Related Code Repository The code used to analyze this data and reproduce the results presented in the paper can be found in a separate repository: https://anonymous.4open.science/r/Needle_In_Factchecking/README.md