Surprising implications of differences in locations versus differences in means

Social science researchers depend on differences in means between experimental and control conditions to draw substantive conclusions. However, an alternative is to use differences in locations. For normal distributions, means and locations are the same, but for skew normal distributions, means and locations are different. If a difference in means and locations are similar, and in the same direction, the resulting substantive story may be similar. However, if a difference in means and locations are dissimilar, especially if they oppose directionally, the resulting substantive story may differ dramatically. We collected 51 data sets from online data repositories to check how often the differences in means versus locations are substantially different or are in different directions. Although the values depend on what one counts, the overall conclusion is that the two types of differences have a larger than trivial chance of disagreeing substantially. We suggest that when researchers report normal statistics (mean and standard deviation), they should report skew normal statistics (location, scale, and shape) too, against the nontrivial chance that the skew normal statistics imply a substantive story in opposition to that implied by the normal statistics. peerReviewed publishedVersion