Fictitious example of a number of proposals submitted for funding and rated by 3 raters for application of the formula by Giraudeau et al.[9] to identify proposals with discordant peer-review ratings (see appendices).

MD: missing data - SD: standard deviation - ICC: intraclass correlation coefficient.The global mean equals 15.7. The inter-proposal SD equals 2.3.(*)Baseline ICC is estimated at 0.366, considering the whole sample. The last column of the table displays re-estimated ICCs using Giraudeau et al. formula once a proposal is discarded. As an example, for proposal 1, the ICC is derived as 0.366 – (0.0018–0.0095), which equals 0.374.