Comparing three groups

For multiple comparisons in analysis of variance, practitioners’ handbooks generally advocate standard methods such as Bonferroni, or an <i>F</i>-test followed by Tukey’s honest significant difference method. These methods are known to be suboptimal compared to closed testing procedures, but improved methods can be complex in the general multi-group set-up. In this note we argue that the case of three-groups is special: with three groups, closed testing procedures are powerful and easy to use. We describe four different closed testing procedures specifically for the three-group set-up. The choice of method should be determined by assessing which of the comparisons are considered primary and which are secondary, as dictated by subject-matter considerations. We describe how all four methods can be used with any standard software.