Generative AI-Assisted Title and Abstract Screening

This dataset supports the study titled "Do it Faster with PICOS: Generative AI-Assisted Systematic Review Screening." The research explores the impact of using an open-source Large Language Model (LLM), Mistral-Nemo-Instruct-2407, to generate structured PICOS (Population, Intervention/Exposure, Comparison, Outcome, Study Design) summaries for title and abstract screening in systematic reviews. The dataset includes article metadata, manually labeled inclusion/exclusion decisions by human reviewers, and LLM-generated structured PICOS summaries.

This dataset can be useful for researchers working on AI-assisted evidence synthesis, systematic review automation, and human-machine collaboration in biomedical literature screening.

The dataset consists of the following columns, capturing key metadata and screening decisions for systematic review articles:

Title – The title of the research article.
Authors – List of authors for the article.
Journal – Name of the journal where the article was published.
Year – Year of publication.
Volume – Journal volume number.
Issue – Journal issue number.
DOI – Digital Object Identifier (DOI) for the article.
Abstract – The abstract of the research article.
PICOS Summary – Structured summary generated by the LLM
Gold_std dataset – Ground truth classification (Include/Exclude) based on expert review.
A1_reviewer – Screening decision by Reviewer A1 (LLM-assisted, less experienced).
A2_reviewer – Screening decision by Reviewer A2 (Traditional, less experienced).
B1_reviewer – Screening decision by Reviewer B1 (LLM-assisted, more experienced).
B2_reviewer – Screening decision by Reviewer B2 (Traditional, more experienced).