Replication Data for: The Sensitivity of Sensitivity Analysis

This article evaluates the reliability of sensitivity tests (Leamer 1978). Using Monte Carlo methods we show that, first, the definition of robustness exerts a large influence on the robustness of var¬iables. Second and more importantly, our results also demonstrate that inferences based on sen¬sitivity tests are most likely to be valid if determinants and confounders are almost uncorrelated and if the variables included in the true model exert a strong influence on outcomes. Third, no definition of robustness reliably avoids both false positives and false negatives. We find that for a wide variety of data-generating processes, rarely used definitions of robustness perform better than the frequently used model averaging rule suggested by Sala-i-Martin. Fourth, our results also suggest that Leamer’s extreme bounds analysis and Bayesian model averaging are extremely un¬likely to generate false positives. Thus, if based on these inferential criteria a variable is robust, it is almost certain to belong into the empirical model. Fifth and finally, we also show that research¬ers should avoid drawing inferences based on lack of robustness.

本文对灵敏度检验（sensitivity tests，Leamer 1978）的可靠性展开评估。我们采用蒙特卡洛（Monte Carlo）方法开展研究，结果显示：其一，稳健性（robustness）的定义对变量的稳健性具有显著影响。其二，也是更为关键的一点：研究结果表明，当解释变量与混杂变量近乎无相关，且真实模型中纳入的变量对被解释变量具有显著影响时，基于灵敏度检验得到的推断最具有效性。其三，不存在任何一种稳健性定义能够同时可靠地规避假阳性（false positive）与假阴性（false negative）错误。我们发现，在多种数据生成过程（data-generating process）场景下，极少被使用的稳健性定义的表现，优于萨拉-伊-马丁（Sala-i-Martin）提出的常用模型平均准则。其四，研究结果还显示，Leamer提出的极端边界分析（extreme bounds analysis）与贝叶斯模型平均（Bayesian model averaging）几乎不可能产生假阳性错误。因此，若基于上述推断准则认定某变量具备稳健性，则该变量几乎必然可纳入实证模型。其五，也是最后一点：我们还证实，研究者应避免基于稳健性缺失的结论进行推断。