ChatGPT as Economics Tutor: Capabilities and Limitations

The dataset comprises responses generated by ChatGPT using three different models (GPT-3.5, GPT-4o, and o1preview) evaluated for their effectiveness as an automated tutor in economics education at universities. The dataset focuses on two key use cases: Explanations of 56 Basic Economic Concepts Answers and Explanations to 25 Multiple-Choice Questions The concepts and questions were sourced from CORE Econ’s The Economy 1.0 textbook. The selected content includes foundational ideas like "Opportunity Costs" and "Aggregate Demand," as well as more advanced topics such as "Asymmetric Information" and "Economic Rent." Responses were generated using standardized prompts that simulate student interactions with ChatGPT. Each response was evaluated using a detailed marking grid that included both problem-specific and response-specific indicators—such as accuracy, scope, error types. A moderation process was applied to ensure reliability, with disagreements resolved through discussion. The final dataset consolidates all model outputs, and their evaluations. It is suitable for analyzing the pedagogical potential and limitations of large language models in educational contexts. Further, all scripts used to evaluate the responses and perform the statistical analysis are included. For more detail see the paper: Brose, Natalie, Christian Spielmann, and Christian Tode. ChatGPT as Economics Tutor: Capabilities and Limitations. School of Economics, University of Bristol, UK, 2025.