Bootstrapping Goodness-of-Fit Statistics for Sparse Categorical Data - Results of a Monte Carlo Study

Analysis of categorical data in educational measurement or psychometrics often faces the problem of how to test model fit for a questionnaire with many items. In this situation, most observed response patterns will be unique and many out of the set of possible response patterns will not be observed at all. The "classical" approach of evaluating goodness-of-fit statistics like the Pearson X^2 or the likelihood-ratio G^2 by means of a χ^2 distribution is not appropriate in these cases, as the data are sparse and the expected frequencies are very low (Fienberg, 1979; Koehler & Larntz, 1980). Performing the bootstrap (Efron, 1979) instead was suggested by many authors (Aitkin, Anderson, & Hinde, 1981; Collins, Fidler, Wugalter, & Long, 1993; Langeheine, Pannekoek, & van de Pol, 1996) as an approach to solving the problem. The bootstrap is supposed to produce a rough approximation of the goodness-of-fit statistics unknown distribution. Langeheine et al. (1996) have shown that the bootstrap works fine for small contingency tables, but for sparse tables, different conclusions regarding the fit of a model can arise if more than one statistic is tested in the bootstrap procedure. Results of a Monte Carlo study focusing on bootstrapping different goodness-of-fit statistics are presented in this paper. The four statistics examined are the Pearson X^2, the Cressie-Read CR(2/3) /Cressie & Read, 1984), the likelihood-ratio G^2and the Freeman-Tukey FT statistics (see Read & Cressie, 1988). The results presented here imply that the parametric bootstrap can be used for analyzing goodness-of-fit, even if the data are very sparse, at least with some of the examined statistics. An explanation based on the behavior of the power-divergence (Cressie & Read, 1988 [5]) statistics as well as practical recommendations for using the bootstrap are given in this paper. unknown publishedVersion