"AI as an Ally?" : AI mediation tools to support undergraduates' argumentative skills

Argumentative skills are indispensable both personally and professionally to process complex information (CoI) relating to the critical reconstruction of meaning through critical thinking (CT). This remains a particularly relevant priority, especially in the age of social media and artificial intelligence-mediated information. Recently, the public dissemination of what has been called generative artificial intelligence (GenAI), with the particular example of ChatGPT (OpenAI, 2022), has made it even easier today to access and disseminate information, written or not, true or not. New tools are needed to critically address post-digital information abundance. 

In this context, argumentative maps (AMs), which are already used to develop argumentative skills and critical thinking, are studied for multimodal and dynamic information visualization, comprehension, and reprocessing. In this regard, the entry of generative AI into university classrooms proposes a novel scenario of multimodality and technological dynamism. 

Building on the Vygotskian idea of mediation and the theory of "dual stimulation" as applied to the use of learning technologies, the idea was to complement AMs with the introduction of a second set of stimuli that would support and enhance individual activity: AI-mediated tools. With AMs, an attempt has been made to create a space for understanding, fixing, and reconstructing information, which is important for the development of argumentative skills. On the other hand, by arranging forms of critical and functional interaction with ChatGPT as an ally in understanding, reformulating, and rethinking one's argumentative perspectives, a new and comprehensive argumentative learning process has been arranged, while also cultivating a deeper understanding of the artificial agents themselves. 

Our study was based on a two-group quasi-experiment with 27 students of the “Research Methods in Education” course, to explore the role of AMs in fixing and supporting multimodal information reprocessing. In addition, by predicting the use of the intelligent chatbot ChatGPT, one of the most widely used GenAI technologies, we investigated the evolution of students' perceptions of its potential role as a “study companion” in information comprehension and reprocessing activities with a path to build a good prompt. 

Preliminary analyses showed that in both groups, AMs supported the increase in mean CoI and CT levels for analog and digital information. However, the group with analog texts showed more complete reprocessing.The interaction with the chatbot was analyzed quantitatively and qualitatively, and there emerged an initial positive reflection on the potential of ChatGPT and increased confidence in interacting with intelligent agents after learning the rules for constructing good prompts.

This Zenodo record follows the full analysis process with R (https://cran.r-project.org/bin/windows/base/ ) and Nvivo (https://lumivero.com/products/nvivo/) composed of the following datasets, script and results:

1. Comprehension of Text and AMs Results - Arg_G1.xlsx & Arg_G2.xlsx

2. Opinion and Critical Thinking level - Opi_G1.xlsx & Opi_G2.xlsx

3. Data for Correlation and Regression - CorRegr_G1.xlsx & CorRegr_G2.xlsx

4. Interaction with ChatGPT - GPT_G1.xlsx & GPT_G2.xlsx

5. Descriptive and Inferential Statistics Comprehension and AMs Building - Analysis_RES_Comprehension.R

6. Descriptive and Inferential Statistics Opinion and Critical Thinking level - Analysis_RES_Opinion.R

7. Correlation and Regression - Analysis_RES_CorRegr.R

8. Descriptive and Inferential Statistics Interaction with ChatGPT - Analysis_RES_ChatGPT.R

9. Sentiment Analysis - Sentiment Analysis_G1.R & Sentiment Analysis_G2.R

10. Vocabulary Frequent words - Vocabulary.csv

11. Codebook qualitative Analysis with Nvivo (Codebook.xlsx)

12. Results Nvivo Analysis G1 - Codebook - ChatGPT2 G1.docx

13. Results Nvivo Analysis G2 - Codebook - ChatGPT2 G2.docx

Any comments or improvements are welcome!