On research execution and replication: The case of automated feedback across different countries

Contains the main data file, Rmd script, and html output of the Rmd script. Article abstract:We report on an experiment designed to investigate the effects of automated feedback on learning in undergraduate mathematics. The original experiment was carried out in England and New Zealand under laboratory conditions, and we also conducted a conceptual replication of the experiment in an authentic didactical setting in Kenya. The research focus was to compare the impact of three feedback conditions (none, accuracy, accuracy+elaborated) on student learning. Specifically, we were interested in how feedback impacts on a robust misconception that a definite integral measures a geometrical area captured between a function and the x-axis. Our main finding was that feedback did not significantly impact post-test scores, and this finding was replicated. We found that the overall patterns of results across four parts of the experimental design (pre-test, priming items, intervention items, post-test) were replicated. However, there were differences in the details of the post-test scores across the two experiments, and in the open-text comments left by some participants. We consider the implications for our understanding of misconceptions and feedback design, and discuss the added value of replication in our research.

本数据集包含主数据文件、Rmd脚本以及该Rmd脚本对应的HTML输出结果。文章摘要：本研究报道一项旨在探究自动化反馈对本科数学学习影响的实验。该实验最初于英国与新西兰的实验室环境下开展，同时我们还在肯尼亚的真实教学场景中完成了该实验的概念性重复验证。本研究的核心目标为对比三种反馈设置（无反馈、仅准确性反馈、准确性+精细化反馈）对学生学习的影响。具体而言，我们重点关注反馈如何影响一种顽固的错误概念：即学生误以为定积分衡量的是函数与x轴之间围成的几何面积。本研究的核心发现为：反馈对后测成绩无显著影响，且该结论得到了重复验证。我们发现，实验设计的四个环节（前测、启动试题、干预试题、后测）的整体结果模式均得到了重复验证。不过，两项实验的后测成绩细节以及部分参与者留下的开放式文本评论存在差异。我们探讨了该结论对理解错误概念及反馈设计的启示，并讨论了重复验证在本研究中的附加价值。