Data for: Inter-rater reliability of risk of bias tools for non-randomized studies

PURPOSE: Currently, there is limited knowledge about the reliability of risk of bias (ROB) tools for assessing internal validity in systematic reviews of exposure and frequency studies. We aimed to identify and then compare the inter-rater reliability (IRR) of six commonly used tools for frequency (Loney scale, Gyorkos checklist, American Academy of Neurology [AAN] tool) and exposure (Newcastle-Ottawa scale, SIGN50 checklist, AAN tool) studies.
METHODS: Six raters independently assessed the ROB of 30 frequency and 30 exposure studies using the 3 respective ROB tools. Articles were rated on a 3-level summary measure of ROB (low, intermediate, or high). We calculated an intraclass correlation coefficient (ICC) for each tool and category of ROB tool. We compared the IRR between ROB tools and tool type by inspection of overlapping ICC 95% CIs and by comparing their coefficients after transformation to Fisher Z values. We assessed criterion validity of the AAN ROB tools by calculating an ICC..., , , # GENERAL INFORMATION

1.     Title of Dataset: Inter-rater Reliability of Risk of Bias Tools for Non-Randomized Studies

2.     Author Information

a.     Principal Investigator Contact Information

Name: Mark Keezer

Institution: Université de Montréal

Address : Centre Hospitalier de l'Université de Montréal, Pavillon R R04-700 1000 Saint-Denis St., Montreal, QC, Canada, H2X 0C1

Email: [[e-mail]](mailto:[e-mail])

b.     Associate or Co-investigator Contact Information

Name: Isabel Kalaycioglu

Institution: Université de Montréal

Email: [[e-mail]](mailto:[e-mail])

3.     Date of data collection (single date, range, approximate date): 2020-2021

4.     Geographic location of data collection: Montréal, Québec, Canada

SHARING/ACCESS INFORMATION

1.     Links to publications that cite or use the data:

Kalaycioglu, I., Rioux, B., Briard, J.N. *et al.* Inter-rater reliability of risk of bias tools for non-r...