AI-Detector Multi-Language Evaluation Dataset

This dataset includes the data relevant to an evaluation of 12 AI-detector tools' performance metrics. The dataset consists of three parts: AI-generated texts in three languages (English, Ukrainian and Russian); Real texts in three languages (English, Ukrainian and Russian); Results and relevant code. The AI-generated texts is a collection of news texts, generated by 48 LLMs divided by 21 folders with the LLM's creator names. Each subfolder is divided by relevant LLM and, where applicable, relevant model versions. Each LLM generated 3 texts in each language and the results are placed in relevant .odt files. For example, AI-Generated Texts/01_OpenAI/GPT/GPT-3.5/News/En-RandomNews-500words.odt is a path to three AI-generated news texts, generated in English by OpenAI's CPT-3.5. The real texts is a collection of news texts, retreived from 9 news agencies from 3 years: 2018, 2019, and 2024. Each year includes three mentioned languages each representing 3 news agencies operating in those languages with each file including 3 news articles from that agency of that year as .odt files. For example: Real texts/2018/EN/BBC_2018.odt represents three articles from the year 2018 by BBC, written in English. Results&Code includes the data.xlsx table, results.txt, Figures folder and Main.py. data.xlsx includes raw predictions and confidence scores of the 12 detectors and is divided into two parts: Sheet1 includes the detectors' predictions and confidence scores for the AI-generated texts, Sheet2 includes the detectors' predictions and confidence scores for the real news articles. Based on this table, Main.py calculates performance metrics and outputs the results either to console or into a file (results.txt in this case). Additionally, relevant figures can be generated (Figures folder). The following is required: scikit-learn, numpy, pandas, openpyxl, and matplotlib. The Figures folder contains 5 subfolders. Detector_Comparisons includes overall score and metric distributions, Negative_Distributions and Positive_Distributions include the individual detector distributions of real and AI-generated texts respectively, PR_Curves and ROC_Curves include overall curves for all of the detectors.

本数据集收录了用于评估12款AI检测工具性能指标的相关数据。数据集包含三个部分：三种语言（英语、乌克兰语、俄语）的AI生成文本；三种语言（英语、乌克兰语、俄语）的真实文本；以及实验结果与配套代码。AI生成文本为新闻文本合集，由48个大语言模型（Large Language Model，LLM）生成，按21个文件夹分类，文件夹以对应大语言模型的开发者命名。每个子文件夹进一步按对应大语言模型及（如适用）模型版本划分。每个大语言模型针对每种语言生成3篇新闻文本，结果存储于对应的.odt文件中。例如，路径AI-Generated Texts/01_OpenAI/GPT/GPT-3.5/News/En-RandomNews-500words.odt指向由OpenAI的GPT-3.5生成的3篇英语AI新闻文本。真实文本为新闻文本合集，采集自9家新闻机构，覆盖2018、2019、2024三个年份。每个年份包含前述三种语言的文本，每种语言对应3家本土新闻机构，每个文件包含该机构当年的3篇新闻文章，以.odt格式存储。例如：Real texts/2018/EN/BBC_2018.odt代表2018年BBC发布的3篇英语新闻文章。结果与代码包含data.xlsx表格、results.txt、Figures文件夹及Main.py。其中data.xlsx收录了12款检测器的原始预测结果与置信度得分，分为两个工作表：Sheet1存储各检测器针对AI生成文本的预测结果与置信度得分，Sheet2存储各检测器针对真实新闻文本的预测结果与置信度得分。Main.py基于该表格计算各项性能指标，并将结果输出至控制台或文件（本案例中为results.txt）。此外，还可生成对应可视化图表（存放于Figures文件夹）。运行代码需依赖以下库：scikit-learn、numpy、pandas、openpyxl及matplotlib。Figures文件夹包含5个子文件夹：Detector_Comparisons存储整体得分与指标分布；Negative_Distributions与Positive_Distributions分别存储针对真实文本与AI生成文本的单检测器得分分布；PR_Curves（精确召回曲线）与ROC_Curves（受试者工作特征曲线）存储所有检测器的整体对应曲线。