Operational characteristics of univariate generalized pairwise comparisons with missing data

Generalized pairwise comparisons is a statistical method that allows comparing two groups of subjects based on a set of hierarchically ordered outcomes. It provides a general measure of treatment effect called the net treatment benefit. The method offers a “natural” way of handling missing data: whenever a comparison of two subjects is not possible for a higher priority outcome due to missingness, the comparison is made using the next outcome in the hierarchy. We have investigated the impact of this naïve way of dealing with missing data on the type-I error probability and power of the test. It appears that the naïve net treatment benefit estimator is biased even under missingness completely at random and fails to guarantee the type-I error probability control at the nominal level. Applying inverse probability weighting reduces the bias but does not provide adequate type-I error probability control. Multiple imputation removes the bias and establishes the control of the type-I error probability.