Arabic_Translated_LAIR_WELFAKE_dataset

Arabic Translated WELFake and LIAR Dataset for Fake News Detection This dataset is a unified Arabic headline corpus constructed for the purpose of binary fake news detection in Arabic. It was produced by translating two established English fake news benchmarks — WELFake and LIAR — into Arabic, followed by a systematic preprocessing and cleaning pipeline, and merging the results into a single resource. Source Datasets The WELFake dataset (Verma et al., 2021) originally contains 72,134 English news articles aggregated from four sources (Kaggle, McIntire, Reuters, and BuzzFeed Political), labeled as fake (0) or real (1). Only the headline (title) field was retained for this work, discarding full article text. The LIAR dataset (Wang, 2017) contains 12,745 short political statements collected from PolitiFact, originally annotated on a six-level veracity scale. The six labels were collapsed into a binary scheme: a true class (half-true, mostly-true, true) and a false class (pants-fire, false, barely-true). Translation All English headlines and statements were translated into Arabic using the Google Translate API via the googletrans Python library. Due to throughput constraints, WELFake was processed in 72 chunk-batches of approximately 1,000 headlines each. For LIAR, the NLLB-200 distilled-600M multilingual model (facebook/nllb-200-distilled-600M) was used as a complementary translation backbone whenever Google Translate produced unstable outputs. Preprocessing Pipeline Both datasets were subjected to an identical Arabic text cleaning pipeline consisting of: (1) URL and hyperlink removal; (2) Arabic diacritic (Tashkeel) removal; (3) Tatweel (kashida) removal; (4) normalization of Arabic letter variants (e.g., أ / إ / آ → ا, ى → ي, ة → ه); (5) removal of Latin-script tokens; (6) removal of Western-Arabic and Eastern-Arabic-Indic digits; (7) removal of all characters outside the Unicode Arabic block; (8) collapsing of repeated characters (e.g., جدااا → جدا); (9) Arabic stop word removal using a curated list covering prepositions, conjunctions, pronouns, demonstratives, and high-frequency reporting verbs; and (10) whitespace normalization and trimming. At the row level, translation-error markers (خطا في الترجمه), empty or single-character instances, exact duplicates, and label-conflicting instances (same text with both fake and real labels) were removed. Final Dataset Composition After merging and applying cross-dataset deduplication and conflict removal, the final unified corpus comprises 74,559 instances. File Structure The dataset is provided as a single UTF-8 encoded CSV file (Arabic_Translated_Welfak_and_Liar_Dataset.csv) with three columns: text — the preprocessed Arabic headline or statement label — binary class label: 1 = Fake, 0 = Real source — origin dataset: WELFake or LIAR用于虚假新闻检测的阿拉伯语翻译版WELFake与LIAR数据集 本数据集为面向阿拉伯语二元虚假新闻检测任务构建的统一阿拉伯语标题语料库。其制作流程为：将两个成熟的英文虚假新闻基准数据集——WELFake与LIAR——翻译为阿拉伯语，随后通过系统化的预处理与清洗流程，最终将处理后的数据合并为单一资源。 源数据集 WELFake数据集（Verma等人，2021）原本包含72134篇英文新闻文章，数据聚合自四个来源：Kaggle、McIntire、路透社（Reuters）与BuzzFeed政治频道，标签分为虚假（0）与真实（1）两类。本研究仅保留标题字段，舍弃完整新闻文本。 LIAR数据集（Wang，2017）包含12745条来自PolitiFact的简短政治声明，原始标注采用六级真实性评分体系。本次研究将六级标签合并为二元分类方案：真实类（半真实、大部分真实、真实）与虚假类（完全虚假、虚假、勉强真实）。 翻译流程 所有英文标题与声明均通过googletrans Python库调用谷歌翻译API（Google Translate API）完成阿拉伯语翻译。受限于吞吐量限制，WELFake数据集以约1000条标题为一批，共分为72个批次进行处理。对于LIAR数据集，当谷歌翻译输出不稳定时，采用NLLB-200 distilled-600M多语言模型（facebook/nllb-200-distilled-600M）作为互补翻译主干模型。 预处理流程 两个数据集均采用统一的阿拉伯语文本清洗流程，具体包括：(1) 移除URL与超链接；(2) 移除阿拉伯语变音符号（Tashkeel）；(3) 移除Tatweel（连字符扩展符，kashida）；(4) 阿拉伯语字母变体归一化（例如：أ/إ/آ → ا， ى → ي， ة → ه）；(5) 移除拉丁字母Token；(6) 移除西阿拉伯数字与东阿拉伯-印度数字；(7) 移除Unicode阿拉伯语区块之外的所有字符；(8) 合并重复字符（例如：جدااا → جدا）；(9) 采用经过精选的词表移除阿拉伯语停用词，该词表涵盖介词、连词、代词、指示代词与高频报道动词；(10) 空白字符归一化及首尾去空格。 在样本层面，需移除翻译错误标记（خطا في الترجمه）、空样本、单字符样本、完全重复样本以及标签冲突样本（同一文本对应虚假与真实两类标签）。 最终数据集构成 经过合并、跨数据集去重与冲突处理后，最终的统一语料库共包含74559条样本。 文件结构 本数据集以单个UTF-8编码的CSV文件（Arabic_Translated_Welfak_and_Liar_Dataset.csv）形式提供，包含三列： text —— 预处理后的阿拉伯语标题或声明 label —— 二元分类标签：1 = 虚假，0 = 真实 source —— 原始数据集来源：WELFake或LIAR