Full BLUFF-1000 dataset and evaluation scripts

Retrieval-augmented generation (RAG) systems often fail to adequately modulate their linguistic certainty when evidence deteriorates. This gap in how models respond to imperfect retrieval is critical for the safety and reliability of a real-world RAG system. To address this gap, we propose \textbf{BLUFF-1000}, a benchmark systematically designed to evaluate how large language models (LLMs) manage linguistic confidence under conflicting evidence to simulate poor retrieval. We created a novel dataset, introduced two novel metrics, calculated full metrics quantifying faithfulness, factuality, linguistic uncertainty, and calibration, and finally conducted experiments on 7 LLMs on the benchmark, measuring their uncertainty awareness and general performance. Our findings uncover a fundamental misalignment between linguistic expression of uncertainty and source quality across seven state-of-the-art RAG systems. We recommend that future RAG systems incorporate uncertainty-aware methods to transparently convey confidence throughout the system.