Using a "sledgehammer" approach to increase systems thinking with a brief manipulation

Systems thinking is a skill that is essential to understanding and taking
effective action on complex challenges such as climate change. This
research evaluated whether systems thinking could be increased with a
brief intervention. Participants (N = 678) recruited from Amazon
Mechanical Turk all completed the Systems Thinking Scale (Randel &
Stroink, 2018), which was used as a covariate. Participants were then
randomly assigned to one of four conditions. Some participants (n = 165)
watched an entertaining 5-minute video describing systems thinking with a
real-life example (Cats in
Borneo, https://www.youtube.com/watch?v=17BP9n6g1F0). Others (n =
174) watched this video, read a definition of systems thinking, and were
asked to engage in systems thinking while completing a survey. This was
designed to be a "sledgehammer" condition, in which we made our
manipulation as heavy-handed as possible. A third (control) condition (n =
167) watched a video about how to fold a fitted sheet. A final control
condition (n = 172) watched no video. All participants completed a survey
that included nine different measures that capture different aspects of
systems thinking. Despite large sample sizes and multiple
operationalizations of systems thinking, support for the efficacy of our
brief intervention was weak at best. Those who watched the systems
thinking video scored significantly higher on one self-report measure of
the extent to which people perceived themselves to be part of local
social, economic, and ecological systems. Watching the systems thinking
video also marginally increased the accuracy with which people correctly
identified positive and negative feedback loops, and the extent to which
participants saw a shot in a pool game impacting the outcome of the game.
However, the control condition performed significantly better than the two
systems thinking video conditions on a stock/flow identification task, and
all other measures showed no differences by condition. We conclude that
increasing systems thinking with a brief manipulation, even one that
defines systems thinking and begs participants to engage in systems
thinking, is not very effective.