Inter-rater agreement in the two setups with different image quality category groups.

Cohens’s weighted kappa was calculated for the grading using 4 image quality categories (Excellent (E)/ Good (G)/ Adequate (A)/ Insufficient (I)) and for the second round of grading using 14 labels. In the latter, the same 4 categories were compiled as in the first round of grading (E/G/A/I). Cohens’s weighted kappa was also determined with E and G merged for both grading rounds [(E+G)/A/I]. To assess the agreement when distinguishing between poor quality images both in the first and second round, Cohen’s weighted kappa was calculated with two merged groups (E and G vs. A and I). The kappa values are presented as medians (interquartile range) for all the graders and for both groups (medical, non-medical).