Comparison of Pre- and Post-semester FCI Scores to Evaluate the Effects of an Online Educational Software on Student Learning

Using the Force Concept Inventory, students within the treatment (n=21) and control (n=23) groups of an engineering mechanics course at UW – Stout were evaluated both prior to treatment and after treatment. The results of this evaluation were interpreted based on the gain for each student as a percentage of maximum possible gain. This metric allows for a more independent analysis among “weaker” and “stronger” students. The results of this interpretation showed that for the treatment group the average gain was 14.0 % (22.1 % if considering only students with positive net change). Similarly, for the control group the average gain was 5.8 % (15.6 % if considering only students with positive net change). Treatment.png shows the plot of post- versus pre-test scores for the treatment group while control.png presents the same for the control group. Each data point represents a student’s pair of scores while the line corresponds with no change in score. The further a data point is from the line, the larger the gain in that student’s score. It is apparent that for the treatment group, larger gains were realized for students spanning from the lowest scoring students to the highest. For the control group, the majority of the large gains were observed with the “weaker” students while the higher performing students actually scored the same or worse on the post-test. These results would seem to suggest that the use of the system benefited a wider selection of the students in the group while the more traditional approach showed greater gains for the “weaker” students with a possible negative impact on the “stronger” students.本研究采用力概念清单（Force Concept Inventory），对威斯康星大学斯托特分校（UW – Stout）工程力学课程的实验组（n=21）与对照组（n=23）学生，分别于教学干预前及干预后开展测评。本次测评结果以每名学生的得分提升率（即最大可能提升百分比）作为解读依据。该指标可实现对“学业薄弱”与“学业优秀”学生群体的独立分析。解读结果显示：实验组学生的平均提升率为14.0%（若仅统计净得分呈正向变化的学生，则平均提升率为22.1%）；对照组学生的平均提升率为5.8%（若仅统计净得分呈正向变化的学生，则平均提升率为15.6%）。Treatment.png展示了实验组的后测得分与前测得分的散点分布关系，control.png则呈现了对照组的对应结果。每个数据点代表一名学生的前后测得分组合，对角线代表得分无变化的基准线。数据点距离该对角线越远，代表该学生的得分提升幅度越大。显而易见，实验组中从低分群体到高分群体的学生均实现了较大幅度的得分提升。而对照组中，多数大幅得分提升均集中于“学业薄弱”学生群体，学业表现更优秀的学生在后测中的得分反而与前测持平甚至有所下滑。上述结果表明，本研究所采用的教学系统使该课程中更多学生获益，而传统教学模式虽让“学业薄弱”学生获得了更显著的提升，却可能对“学业优秀”学生产生负面影响。