EvidenceGRADEr dataset

What is EvidenceGrader?

Assessment of the quality of medical evidence is a critical step in the systematic review of clinical evidence.
We aim to develop and evaluate a machine-learning system to enable automation of this step. The system should provide
an overall quality rating for each outcome as well as finer-grained justification for different quality criteria according to
the GRADE formalisation framework. Our goal is to work towards evaluating the quality of bodies of evidence (BoE)
for a specific clinical question, rather than assessing the quality of individual primary studies.

The dataset

We have built a dataset of 13,459 instances built from quality-related information available in Cochrane Database of Systematic Reviews. To obtain the full dataset, a request needs to be made first through: https://community.cochrane.org/form/cochrane-library-data-request, upon which we can share all dataset files.

The file fold_ids.json available here contains only the identifiers of summary-of-findings tables from which we extracted the data instances. One instance corresponds to one outcome per summary-of-finding table (there can be more than one outcome). The entire collection is split into 10 folds.