Data underlying the writing assessment study on depth perception representation during rating

The research that the current dataset underlies proposes to use depth perception to represent raters’ decision when evaluating ESL essays, as an alternative medium to conventional form of numerical scores. The researchers verified the new method’s accuracy and inter/intra-rater reliability by inviting 24 ESL teachers to perform different representations when rating 60 essays written by Chinese ESL learners. During numerical representation, the raters expressed their evaluation in form of numbers, while with the DP-based method the raters express their evaluation by marking distances with nail tags on a wood ruler with hidden scale marks. Then the researchers translated the distance results into numbers, and compared the accuracy and inter/intra-rater consistency of the two approaches by referring to these essays’ criteria scores from 8 expert raters. This data file contains rating results that are listed in separate columns, Ea for 30 random essays and Eb for the other 30 essays, Groups A for 12 raters using numer representation mathod and Group B for the other 12 raters using DP representation method. The scoring results by 8 experts are also included in this data file.

本数据集所支撑的该项研究提出，采用深度感知（Depth Perception, DP）来表征评分者在评价英语作为第二语言（English as a Second Language, ESL）作文时的决策过程，以此作为传统数值分数形式的替代方案。研究人员邀请24名ESL教师对60篇中国ESL学习者撰写的作文开展评分实验，通过两种不同的表征方式验证了新方法的准确性以及评分者间信度与评分者内部信度：在数值表征组中，评分者以数字形式给出评价；而在基于DP的方法组中，评分者通过在带有隐藏刻度的木质标尺上用钉状标签标记距离来表达评价结果。随后研究人员将距离测量结果转换为数值，并参考8名专家评分者给出的该批作文的基准评分，对比了两种方法的准确性以及评分者间、评分者内部一致性。本数据集包含以分列形式呈现的评分结果：其中Ea对应30篇随机抽取的作文，Eb对应剩余30篇作文；Group A为12名采用数值表征方法的评分者组，Group B为另外12名采用DP表征方法的评分者组。数据集同时收录了8名专家的评分结果。