Replication Data for: No Evidence that Measuring Moderators Alters Treatment Effects

As survey experiments have become more common in political science, so too have efforts to identify who is most responsive to a treatment. By necessity, moderation experiments frequently rely on observed, rather than manipulated moderators, such as partisan identity or racial attitudes. These designs have led to an ongoing debate about where to measure moderators – immediately prior to the treatment, after the treatment, or in a prior wave of a panel survey. Each design choice has its downsides and detractors. Measuring a moderator after the treatment opens the possibility of posttreatment bias. Measuring it prior to the experiment may create priming effects. And panel studies are costly and sometimes infeasible. We contribute to this debate by systematically studying whether measuring moderators prior to an experiment affects the results. Across six different experiments, each involving one of the most commonly used moderators, we find no evidence of priming effects. Additionally, we find no evidence that the distance between a moderator and an experiment within a survey affects the results. Our findings thus help resolve the debate, suggesting that the pretreatment measurement of a moderator often poses little threat to the inferences drawn from an experiment. We conclude with advice on designing well-powered moderation experiments.