Replication Data for Navigating the Mismeasurement of Intermediary Variables in Message-Based Experiments

Researchers frequently deliver treatments through messages, as in many audit and get-out-the-vote (GOTV) experiments. These message-based experiments often hinge on intermediary variables -- actions subjects must take to actually receive the treatment or control embedded in a message. In particular, whether subjects open the message is a crucial intermediary step, which can serve as a condition for estimating treatment effects on subsequent behaviors or as an outcome of interest in its own right. Yet researchers often measure opens with error -- specifically, the misclassification of some openers as non-openers -- most notably in the case of messages via email. This paper derives the bias that such measurement error introduces into causal effect estimation. Despite this bias, we show that standard approaches still recover interpretable bounds on effects for well-defined subgroups. We also show how researchers can assess the sensitivity of their inferences to different degrees of mismeasurement. By clarifying the causal conclusions warranted by message-based experiments and proposing methodological improvements, this paper provides guidance for applied practice in message-based experiments conducted through email and through other communication technologies.