Test scores’ robustness to scaling: The scale_transformation command

Social scientists frequently rely on the cardinal comparability of test scores to assess achievement gaps between population subgroups and their evolution over time. This approach has been criticized because of the ordinal nature of test scores and the sensitivity of results to order-preserving transformations that are theoretically plausible. Bond and Lang (2013, Review of Economics and Statistics 95: 1468–1479) document the sensitivity of measured ability to scaling choices and develop a method to assess the robustness of changes in ability over time to scaling choices. In this article, I present the scale_transformation command, which expands the Bond and Lang (2013) method to more general cases and optimizes their algorithm to work with large datasets. The command assesses the robustness of an achievement gap between two subgroups to any arbitrary choice of scale by finding bounds for the original gap estimation. Additionally, it finds scale transformations that are very likely and unlikely to benchmark against the results obtained. Finally, it also allows the user to measure how much gap growth coefficients change when including controls in their specifications.

社会科学家通常依托测验分数的基数可比性（cardinal comparability），评估不同人口子群体间的成就差距及其随时间的演变趋势。该方法广受批评，原因在于测验分数本身具备序数属性，且研究结果对理论上合理的保序变换（order-preserving transformations）十分敏感。邦德与朗（Bond and Lang，2013，《经济与统计评论（Review of Economics and Statistics）》第95卷：1468–1479页）指出，测度得到的能力对量表选择（scaling choices）具有敏感性，并提出了一种方法，用以评估能力随时间的变化对量表选择的稳健性。本文中，笔者推出了scale_transformation命令，该命令将邦德与朗（2013）提出的方法拓展至更通用的场景，并对其算法进行优化，使其可适配大规模数据集。该命令通过求解原始差距估计值的边界区间，评估两组人口子群体间的成就差距对任意量表选择的稳健性。此外，该命令还可识别出与所得结果进行基准对比时，极有可能发生与几乎不可能发生的量表变换。最后，该命令还支持用户测算在其模型设定中加入控制变量时，差距增长系数的变化幅度。