A Decade-long Landscape of Advanced Persistent Threats: Longitudinal Analysis and Global Trends

A Decade-long Landscape of Advanced Persistent Threats: Longitudinal Analysis and Global Trends This repository accompanies the paper "A Decade-long Landscape of Advanced Persistent Threats: Longitudinal Analysis and Global Trends", published in the Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS '25). It provides: Curated datasets from the longitudinal study of Advanced Persistent Threat (APT) campaigns across the last decade Visual representations of APT campaigns, including an interactive map and a flow diagram showing relationships between threat actors and target countries Python code to generate the figures included in the paper for full reproducibility _______________________________________________________________________________________________________________ Dataset Overview The repository contains the following collections: Threat Actor Collection Aggregates APT threat actor (TA) information from three curated open-source repositories (TA#1–TA#3): TA#1 MISP Galaxy TA#2: EternalLiberty TA#3: APTmap Each record provides: "Threat Actor": Unique identifier "Other Names": Known aliases "Country": Attributed country of origin "Sponsor": Sponsoring entity Motivation "First seen": First recorded year of activity Technical Report Collection Consolidates metadata on APT technical reports from three open-source repositories (TR#1–TR#3): TR#1: APT & Cybercriminals Campaign Collection TR#2: APTnotes TR#3: Malpedia Metadata fields include: "Date": Publication date Filename Title "Download Url": Source download link Information Retrieved Collection Refined dataset resulting from the extraction and validation of structured information from the reports. Information was obtained through a combination of rule-based, LLM-based, and manual methods: The final dataset after information retrieval and refinement of generated answers. The information was retrieved using rule-based (i.e. IoCParser), LLM-based (i.e. GPT-4-Turbo), and manual retrievals. Rule-based Retrieval IoCParser, a tool designed for processing IoCs from various data sources, was chosen. It focuses on extracting: CVE identifiers MITRE ATT&CK technique IDs YARA rules LLM-based Retrieval After comparative evaluation, GPT-4-Turbo was selected for its highest performance across precision, recall, and F1 score metrics. The model was used to extract: Threat actor attribution Victim country Use of zero-day exploits Initial attack vectors Malware names Targeted sectors Campaign duration Manual Verification Due to the persistent and stealthy nature of APT campaigns, LLM-derived information on attack durations was manually reviewed and validated for accuracy. Visual Representations This repository provides interactive visualizations that complement the findings in the paper: Interactive APT Map A map enabling exploration of APT campaigns by selecting either an attacking or victim country. It presents decade-long historical data including threat actor(s), CVEs, attack vector(s), malware, target sector(s), and estimated duration. Data is dynamically updated using LLM-based retrieval from TR#1. It also integrates a timeline chart linking campaigns to relevant news articles for additional context. Threat Actor - Victim Country Flow Diagram An interactive Sankey-style diagram visualizing the relationships between the top 10 threat actors and the 30 most frequently targeted countries over the past decade. Global Trends The Global Trends directory contains Python scripts for generating the figures presented in the paper. Each script reads from the curated datasets in this repository and outputs a figure in PDF format using the same visual style and parameters as in the published paper. Usage All figure drawing scripts require Python 3.8+ and the following Python packages: pip install pandas numpy altair vl-convert-python seaborn matplotlib After installing the dependencies, you can run a script with: cd "Global Trends" python {desiredCode}.py Running a script will produce a PDF file in the current directory. Font Configuration The figures in the paper use specific fonts. If they are missing, Matplotlib will show "findfont" warnings and fall back to its default fonts. The scripts will still run correctly, and figures will be generated. To suppress the warnings and use the intended fonts on Linux systems, run: sudo apt install msttcorefonts -qq rm -rf ~/.cache/matplotlib On Windows and macOS, installing these fonts can be more complex, and is optional.十年高级持续性威胁全景：纵向分析与全球趋势 本仓库配套发表于2025年ACM SIGSAC计算机与通信安全大会（CCS '25）的论文《十年高级持续性威胁全景：纵向分析与全球趋势》。 本仓库提供以下内容： 1. 涵盖过去十年高级持续性威胁（Advanced Persistent Threat, APT）活动纵向研究的精选数据集 2. APT活动可视化成果，包括交互式地图与展示威胁行动者与目标国家关联关系的流图 3. 用于复现论文中所有图表的Python代码，保障研究可完全复现。 ## 数据集概览 本仓库包含以下数据集集合： ### 威胁行动者数据集（Threat Actor Collection） 从三个精选开源仓库（TA#1–TA#3）汇总APT威胁行动者（Threat Actor, TA）信息： - TA#1：MISP Galaxy - TA#2：EternalLiberty - TA#3：APTmap 每条记录包含以下字段： - "威胁行动者"：唯一标识符 - "别名"：已知别称 - "所属国家"：归因的起源国家 - "资助方"：资助实体 - "攻击动机" - "首次出现时间"：首次记录的活动年份 ### 技术报告数据集（Technical Report Collection） 从三个开源仓库（TR#1–TR#3）整合APT技术报告元数据： - TR#1：APT与网络犯罪活动数据集（APT & Cybercriminals Campaign Collection） - TR#2：APTnotes - TR#3：Malpedia 元数据字段包括： - "发布日期"：出版日期 - "文件名" - "标题" - "下载链接"：来源下载地址 ### 信息提取数据集（Information Retrieved Collection） 该数据集为从报告中提取并验证结构化信息后得到的精炼数据集。 信息提取结合了基于规则、大语言模型（Large Language Model, LLM）与人工三种方法： 最终数据集为经过信息提取与生成结果精炼后的产物。 信息提取采用了基于规则（即IoCParser）、大语言模型（即GPT-4-Turbo）与人工提取三种方式。 #### 基于规则的提取（Rule-based Retrieval） 选用IoCParser——一款专为处理多源数据中的威胁指示器（Indicator of Compromise, IOC）设计的工具。 其提取重点包括： - CVE标识符 - MITRE ATT&CK技术ID - YARA规则 #### 基于大语言模型的提取（LLM-based Retrieval） 经过对比评估后，GPT-4-Turbo因其在精确率、召回率与F1值指标上表现最优而被选中。该模型用于提取以下信息： - 威胁行动者归因 - 受害国家 - 零日漏洞利用情况 - 初始攻击向量 - 恶意软件名称 - 目标行业 - 活动持续时长 #### 人工验证（Manual Verification） 鉴于APT活动具有持久性与隐蔽性特征，模型提取的攻击持续时长信息需经人工审核与验证，以确保准确性。 ## 可视化成果 本仓库提供可辅助论文研究发现的交互式可视化工具： 1. **交互式APT地图** 该地图支持通过选择攻击方或受害国家来探索APT活动，涵盖十年历史数据，包括威胁行动者、CVE漏洞、攻击向量、恶意软件、目标行业与估算持续时长。数据通过基于大语言模型对TR#1的检索实现动态更新，同时集成时间线图表，关联活动与相关新闻文章以提供额外背景信息。 2. **威胁行动者-受害国家流图** 一款交互式桑基（Sankey）风格图表，可视化展示过去十年间排名前10的威胁行动者与被攻击最频繁的30个国家之间的关联关系。 ## 全球趋势模块 全球趋势目录包含用于复现论文中各类图表的Python脚本。 每个脚本均读取本仓库内的精选数据集，并采用与发表论文一致的视觉样式与参数，将图表输出为PDF格式。 ## 使用方法 所有绘图脚本均需Python 3.8及以上版本，并依赖以下Python包： pip install pandas numpy altair vl-convert-python seaborn matplotlib 安装依赖后，可通过以下命令运行脚本： cd "Global Trends" python {目标脚本名称}.py 运行脚本将在当前目录生成PDF文件。 ## 字体配置 论文中的图表使用了特定字体。若系统缺失这些字体，Matplotlib将触发“findfont”警告并回退至默认字体，脚本仍可正常运行并生成图表。 若需消除警告并在Linux系统中使用指定字体，请执行以下命令： sudo apt install msttcorefonts -qq rm -rf ~/.cache/matplotlib 在Windows与macOS系统中，安装此类字体的流程更为复杂，且为可选操作。