The Science of Technology and Human Behavior: Standards, Old and New

Editorial of the Journal of Media Psychology special issue on "Technology & Human Behavior", and meta-analysis of the empirical research published in JMP since 2008.

DATA AVAILABILITY
We were not able to identify a single publication reporting a link to research data in a public repository or the journal’s supplementary materials.

STATISTICAL REPORTING ERRORS
We extracted a total of 1036 NHSTs reported in 98 articles. 129 tests were flagged as inconsistent (i.e., reported test statistics and degrees of freedom do not match reported p-values), of which 23 were grossly inconsistent (the reported p-value is <.05 while the recomputed p-value is >.05, or vice-versa). 41 publications reported at least one inconsistent NHST, and 16 publications reported at least one grossly inconsistent NHST. Thus, a substantial proportion of publications in JMP seem to contain inaccurately reported statistical analyses, of which some might affect the conclusions drawn from them.

STATISTICAL POWER
As in other fields, surveys tend to have healthy sample sizes apt to reliably detect medium to large relationships between variables. The median sample size for survey studies is 327, allowing researchers to detect small bivariate correlations of r=.1 at 44% power (rs=.3/.5 both > 99%).
For (quasi-)experiments, the outlook is a bit different, with a median sample size of 107. Across all types of designs, the median condition size is 30.67. Thus, the average power of experiments published in JMP to detect small differences between conditions (d=.20) is 12% (d=.50 at 49%, d=.80 at 87%).

《媒体心理学杂志》特刊“技术与人类行为”的编者按，以及自2008年以来在该杂志上发表的实证研究的荟萃分析。

数据可用性
我们未能识别出任何单一出版物报告将研究数据链接至公共存储库或期刊的补充材料。

统计报告错误
我们从98篇文章中提取了总共1036个NHSTs（经典假设检验统计）。其中129项测试被标记为不一致（即，报告的测试统计量和自由度与报告的p值不匹配），其中23项严重不一致（报告的p值小于0.05，而重新计算的p值大于0.05，或反之）。41篇出版物报告了至少一个不一致的NHST，16篇出版物报告了至少一个严重不一致的NHST。因此，JMP中相当一部分出版物似乎含有不准确报告的统计分析，其中一些可能会影响据此得出的结论。

统计功效
与其他领域一样，调查通常具有适宜的样本量，足以可靠地检测变量之间中等至大的关系。调查研究的样本量中位数是327，允许研究者以44%的功效（r的双变量相关系数为0.1时）检测到小的双变量相关性（r的双变量相关系数为0.3至0.5时均大于99%）。对于（准）实验，情况略有不同，样本量中位数是107。在所有类型的设计中，条件大小中位数是30.67。因此，JMP中发表的实验的平均功效，用于检测条件之间的小差异（d=0.20）是12%（d=0.50时为49%，d=0.80时为87%）。