Replication data for: Strengthening the Experimenter's Toolbox: Statistical Estimation of Internal Validity

Experiments have become an increasingly common tool for political science researchers over the last decade, particularly laboratory experiments performed on small convenience samples. We argue that the standard normal theory statistical paradigm used in political science fails to meet the needs of these experimenters and outline an alternative approach to statistical inference based on randomization of the treatment. The randomization inference approach not only provides direct estimation of the experimenter's quantity of interest---the certainty of the causal inference about the observed units---but also helps to deal with other challenges of small samples. We offer an introduction to the logic of randomization inference, a brief overview of its technical details, and guidance for political science experimenters about making analytic choices within the randomization inference framework. Finally, we reanalyze data from two political science experiments using randomization tests to illustrate the inferential differences that choosing a randomization inference approach can make.

近十年来，实验已成为政治学研究者愈发常用的研究工具，其中尤以基于小型便利样本开展的实验室实验为甚。我们指出，政治学研究中沿用的标准正态理论统计范式（normal theory statistical paradigm）无法适配这类实验的研究需求，随后我们提出一种基于处理随机化的统计推断替代路径，并对其展开系统阐述。随机化推断（randomization inference）方法不仅可以直接估算研究者的关注量——即针对观测单元的因果推断置信度——同时也能助力解决小型样本面临的其他研究难题。我们还就随机化推断的逻辑内涵展开了介绍，简要概述了其技术细节，并为政治学实验研究者提供了在随机化推断框架内进行分析选择的实操指引。最后，我们运用随机化检验（randomization tests）方法对两项政治学实验的原始数据进行了重新分析，以此说明选用随机化推断方法所带来的推断结果差异。