Why do Experts Miss AI's Errors? Evidence from a Randomized Labeling Experiment

Using a randomized experiment, we test how educators revise grades labeled as either algorithm- or human-generated. Each teacher saw identical student work paired with an intentionally incorrect score; we varied (i) whether the correct answers caused the incorrect score to be harsh or lenient and (ii) the stated source of the mark. The outcome, the grading fairness gap, is the distance between teachers’ revised marks and the objective grade. Under a harsh recommendation, the gap was 22% larger with an AI label; in the lenient case, the fairness gap under AI and human labels was statistically indistinguishable. Mediation analysis shows that, in the harsh case, higher attributions of ability and responsibility to the algorithm transmit no less than half of the effect, whereas weaker attributions in the lenient case trigger stricter corrections. Thus, acceptance of algorithmic advice hinges on error direction and inferred credibility, rather than on artificial intelligence alone.