LLM-Generated Bibliography

This dataset supports the paper 'Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature'. It contains a collection of bibliography entries generated by LLMs to study the frequency and nature of citation hallucinations.

The dataset includes bibliographic entries generated by a diverse set of Large Language Models (LLMs), covering different architectures and scales. Specifically, the following models were evaluated:

Microsoft Phi Family: phi:2.7b:4q, phi3.5:3.8b:4q

Meta Llama Family: llama2:7b:4q, llama3.3:70b:4q

Google Gemma Family: gemma:7b:4q, gemma2:9b:4q

Specialized/Other Models: dolphin-mistral:7b:4q, command-r:35b:4q, deepSeek-r1:8b

Experimental Design: Independent Zero-Shot Runs

To ensure the robustness of our findings and account for the inherent stochasticity of LLM outputs, we implemented the following protocol:

Zero-Shot Prompting: All experiments were conducted in a strictly zero-shot setting. No previous examples of correct bibliographies were provided in the prompt, forcing the models to rely entirely on their internal knowledge and pre-training.

Independent Iterations (Suffixes _1, _2, _3): Each model was prompted to generate bibliographies in three independent experimental runs. The numerical suffixes in the column names or filenames (e.g., llama3.3_1, llama3.3_2, llama3.3_3) correspond to these distinct iterations.

Independence of Data Points: Each BibTeX entry and each experimental run is treated as an independent observation. There is no memory or context carry-over between the 1st, 2nd, and 3rd runs of the same model.

If you use the dataset, please reference the paper:

Picazo-Sanchez, P., & Ortiz-Martin, L. (2026). Evaluating the Integrity of LLM-Generated Citations: Prevalence and Risks of Fabricated References in Scientific Literature. Data, 11(5), 122. https://doi.org/10.3390/data11050122