Artificial intelligence (AI) model responses to tax assessment questions at the university second-year level

This study analyses the accuracy of closed-source AI models in responding to second-year taxation assessments at UP. The research question guiding the study was: How accurately can AI models answer taxation assessments from the University of Pretoria and align with official memoranda? To address this, three objectives were pursued: (1) to measure the alignment between AI-generated responses and model answers, (2) to compare the performance of different AI models, and (3) to assess the models’ mastery of the RBT cognitive dimensions. Adopting a pragmatic philosophy, the study employed a mixed-methods design. Quantitative accuracy was tested by benchmarking AI responses against official memoranda, while qualitative analysis coded responses into RBT categories. Data were drawn from the 2024 TXA201 exam and supplementary exam and analysed using a standardised coding framework and predetermined prompts. The findings demonstrate that AI models excel at lower-order cognitive tasks such as remembering, understanding, and applying, but their performance declines with higher-order tasks requiring analysis, evaluation, and creation. The study concludes that AI holds potential as a supplementary tool in taxation education, yet it cannot replace human reasoning, judgment, and contextual interpretation.<br>