MAGE framework testing of ChatGPT4

Evaluation of vulnerabilities of critical thinking skills and values of ChatGPT4 as of October 2023.
The MAGE framework to evaluate six cognitive skills - justify, evaluate, reflect, analyse, interpret and explain. There were eight values of inquiry used - clarity, accuracy, precision, depth, breadth, relevance and significance and coherence. Each document contains one cognitive skill and four appropriate values of inquiry. Four values per cognitive skill were seen as the maximum number that could be effectively evaluated, as well as being likely to be the maximum number of values that would be assessed in a given task.

Each task corresponds to an undergraduate university assessment question, with the precise details altered to maintain the integrity of the original exam (unless publicly available on the UQ eSpace). Due to the limitations of ChatGPT4 and the number of tokens provided, only the “minimally adapted” and “prompt engineered” tests were performed.

It is unknown if these results will be replicable as the technology continues to update.