Extrinsic Convergent Validity Evidence to Prevent Jingle and Jangle Fallacies

In psychology, there have been vast creative efforts in proposing new constructs and developing measures to assess them. Less effort has been spent in investigating construct overlap to prevent bifurcated literatures, wasted research efforts, and jingle-jangle fallacies. For example, researchers could gather validity evidence to evaluate if two measures with the same label actually assess different constructs (jingle fallacy), or if two measures with different labels actually assess the same construct (jangle fallacy). In this paper, we discuss the concept of extrinsic convergent validity, a source of validity evidence demonstrated when two measures of the same construct, or two measures of seemingly different constructs, have comparable correlations with external criteria. We introduce a formal approach to obtain extrinsic convergent validity evidence using tests of dependent correlations and evaluate the tests using Monte Carlo simulations. Also, we illustrate the methods by examining the overlap between the self-control and grit constructs, and the overlap among seven seemingly different measures of the connectedness to nature construct. Finally, we discuss how extrinsic convergent validity evidence supplements other sources of evidence that support validity arguments of construct overlap.

在心理学领域，学者们致力于提出新的理论结构并开发相应的评估方法，然而在探究结构重叠性方面投入的努力相对较少，这导致了文献的分裂、研究资源的浪费以及似是而非谬误的产生。例如，研究者可以收集效度证据以评估带有相同标签的两个测量工具是否实际上评估了不同的结构（铃铛谬误），或者带有不同标签的两个测量工具是否实际上评估了同一结构（铃铛谬误）。在本研究中，我们讨论了外在收敛效度的概念，即当同一结构的两个测量工具，或看似不同结构的两个测量工具与外部标准具有可比的关联性时，所展现出的效度证据来源。我们引入了一种正式的方法，通过依赖相关性的测试来获取外在收敛效度的证据，并使用蒙特卡洛模拟来评估这些测试。此外，我们通过考察自我控制与坚毅结构之间的重叠性，以及自然联系结构七个看似不同的测量工具之间的重叠性，来展示这些方法。最后，我们讨论了外在收敛效度的证据如何补充支持结构重叠性效度论据的其他证据来源。