Using Total Margin of Error to Account for Non-Sampling Error in Election Polls

The potential impact of non-sampling errors on election polls is well known, but measurement has focused on the margin of sampling error. Statisticians have recommended measurement of total survey error by mean square error (MSE), which jointly measures sampling and non-sampling errors. We suggest use of the square root of maximum MSE to measure the total margin of error (TME). We suggest that measurement of TME should be a standard feature in the reporting of polls. Because the exceedingly low response rates commonly obtained by polls is a particularly worrisome source of potential error, we demonstrate how to measure the potential impact of nonresponse by sampled persons using the concept of TME. We show how to measure TME when a pollster lacks any knowledge of the candidate preferences of nonrespondents. We extend the analysis to settings where the pollster has partial knowledge that bounds the preferences of non-respondents. In each setting, we derive a simple poll estimate that approximately minimizes TME—a midpoint estimate—and compare it to a conventional poll estimate. We extend our analysis of nonresponse by sampled persons to address polls with under-coverage of the population and opt-in panels of participants.