main code

This folder contains the Python scripts used to replicate and evaluate the experiments in the paper. The code is organized according to the corresponding figures in the paper. Below is a description of each script and its corresponding figure.Folder Structure1. <b>similarity.py</b><b>Figure 4</b>: This script calculates the similarity between AI-generated debunking texts and official debunking texts.<b>Functionality</b>: It takes the AI-generated debunking texts and the official debunking texts as input and computes similarity scores. These scores are used to evaluate how closely the AI-generated texts match the official ones in terms of content.2. <b>token_deepseek_g.py</b><b>Figure 2(a)</b>: This script visualizes the distribution of token quantities between generated rumors and the refusal to generate rumors, specifically for the <b>DeepSeek-R1</b> model.<b>Functionality</b>: It processes the token counts for both AI-generated rumors and the refusal to generate them, allowing for a comparative analysis of token distribution in these two categories.3. <b>token_detection.py</b><b>Figure 3</b>: This script visualizes the distribution of tokens between correctly identified rumors and misidentified rumors.<b>Functionality</b>: It processes the token counts for rumors correctly classified as such and those that were misclassified, highlighting the token distribution in each category for the rumor detection model.4. <b>token_qwq_g.py</b><b>Figure 2(b)</b>: This script visualizes the token distribution between generated rumors and the refusal to generate rumors, but for the <b>qwq-32b</b> model.<b>Functionality</b>: It processes the token distribution for AI-generated rumors and the refusal to generate rumors using the <b>qwq-32b</b> model, providing an additional comparison across different models.5. <b>readability.py</b><b>Figure 5</b>: This script calculates the readability of the generated debunking texts, comparing AI-generated debunking texts with official debunking texts.<b>Functionality</b>: The script computes readability scores using the <b>Flesch Reading Ease</b> formula for the debunking texts produced by both the <b>DeepSeek-R1</b> and <b>qwq-32b</b> models, as well as the official debunking texts.Additional Figures<b>Figure 1 and Figure 6</b><b>Figure 1</b>: This figure represents the overall statistical analysis derived directly from the datasets in the <b>DeepSeek-R1 Generation</b> and <b>qwq-32b Generation</b> subfolders. The figures were plotted using Microsoft PowerPoint based on the raw statistical results from these two datasets.<b>Figure 6</b>: Similarly, Figure 6 was created based on sentiment analysis results directly obtained from the <b>sentiment_analysis_R.json</b> and <b>Q_sentiment_analysis.json</b> files. The plotting was done using Microsoft PowerPoint, with statistical data from these files.<br>UsageDownload Main DataBefore running the scripts in this folder, please download the data files from the <b>Main Data</b> folder. These data files are essential for processing and generating results. You can find the required data in the following folders:<b>deepseek-r1-generation</b><b>deepseek-r1-detection</b><b>deepseek-r1-debunking</b><b>qwq-32b-generation</b><b>qwq-32b-detection</b><b>qwq-32b-debunking</b><b>deepseek-v3-detection</b>Make sure to download these files before proceeding to the next steps.Step 2: Run ScriptsOnce you have the data, you can run the scripts individually. Each script processes the data and generates outputs such as JSON files or plots. The results correspond to the figures in the paper.AcknowledgementsThe data used in these experiments (<b>FakeNewsNet</b> and <b>Twitter1516</b>) are publicly available datasets.<br>