Dataset and Source Code for the Paper: A Framework for Developing Strategic Cyber Threat Intelligence from Advanced Persistent Threat Analysis Reports Using Graph-Based Algorithms

Here are the data set and source code related to the paper: "A Framework for Developing Strategic Cyber Threat Intelligence from Advanced Persistent Threat Analysis Reports Using Graph-Based Algorithms"

1- aptnotes-downloader.zip : contains source code that downloads all APT reports listed in https://github.com/aptnotes/data and https://github.com/CyberMonitor/APT_CyberCriminal_Campagin_Collections

2- apt-groups.zip : contains all APT group names gathered from https://docs.google.com/spreadsheets/d/1H9_xaxQHpWaa4O_Son4Gx0YOIzlcBWMsdvePFX68EKU/edit?gid=1864660085#gid=1864660085 and https://malpedia.caad.fkie.fraunhofer.de/actors and https://malpedia.caad.fkie.fraunhofer.de/actors

3- apt-reports.zip : contains all deduplicated APT reports gathered from https://github.com/aptnotes/data and https://github.com/CyberMonitor/APT_CyberCriminal_Campagin_Collections

4- countries.zip : contains country name list.

5- ttps.zip : contains all MITRE techniques gathered from https://attack.mitre.org/resources/attack-data-and-tools/

6- malware-families.zip : contains all malware family names gathered from https://malpedia.caad.fkie.fraunhofer.de/families

7- ioc-searcher-app.zip : contains source code that extracts IoCs from APT reports. Extracted IoC files are provided in report-analyser.zip. Original code repo can be found at https://github.com/malicialab/iocsearcher

8- extracted-iocs.zip : contains extracted IoCs by ioc-searcher-app.zip

9- report-analyser.zip : contains source code that searchs APT reports, malware families, countries and TTPs. I case of a match, it updates files in extracted-iocs.zip.

10- cti-transformation-app.zip : contains source code that transforms files in extracted-iocs.zip to CTI triples and saves into Neo4j graph database.

11- graph-db-backup.zip : contains volume folder of Neo4j Docker container. When it is mounted to a Docker container, all CTI database becomes reachable from Neo4j web interface. Here is how to run a Neo4j Docker container that mounts folder in the zip:

docker run -d --publish=7474:7474 --publish=7687:7687 --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/data:/data --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/plugins:/plugins --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/logs:/logs --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/conf:/conf --env 'NEO4J_PLUGINS=["apoc","graph-data-science"]' --env NEO4J_apoc_export_file_enabled=true --env NEO4J_apoc_import_file_enabled=true --env NEO4J_apoc_import_file_use__neo4j__config=true --env=NEO4J_AUTH=none neo4j:5.13.0

web interface: http://localhost:7474

username: neo4j

password: neo4j

本数据集及配套源代码对应论文《基于图算法从高级持续性威胁分析报告构建战略网络威胁情报的框架》（A Framework for Developing Strategic Cyber Threat Intelligence from Advanced Persistent Threat Analysis Reports Using Graph-Based Algorithms）。

1. aptnotes-downloader.zip：包含用于下载https://github.com/aptnotes/data 与 https://github.com/CyberMonitor/APT_CyberCriminal_Campagin_Collections 中所列全部高级持续性威胁（Advanced Persistent Threat, APT）报告的源代码。

2. apt-groups.zip：包含从以下来源收集的全部APT组织名称：https://docs.google.com/spreadsheets/d/1H9_xaxQHpWaa4O_Son4Gx0YOIzlcBWMsdvePFX68EKU/edit?gid=1864660085#gid=1864660085、https://malpedia.caad.fkie.fraunhofer.de/actors 以及 https://malpedia.caad.fkie.fraunhofer.de/actors。

3. apt-reports.zip：包含从https://github.com/aptnotes/data 与 https://github.com/CyberMonitor/APT_CyberCriminal_Campagin_Collections 中收集的全部去重后的APT报告。

4. countries.zip：包含国家名称列表。

5. ttps.zip：包含从https://attack.mitre.org/resources/attack-data-and-tools/ 中收集的全部MITRE攻击技术。

6. malware-families.zip：包含从https://malpedia.caad.fkie.fraunhofer.de/families 中收集的全部恶意软件家族名称。

7. ioc-searcher-app.zip：包含用于从APT报告中提取威胁指示符（Indicator of Compromise, IoC）的源代码。提取得到的IoC文件已包含在report-analyser.zip中。该源代码的原始仓库地址为https://github.com/malicialab/iocsearcher。

8. extracted-iocs.zip：包含由ioc-searcher-app.zip提取得到的IoC文件。

9. report-analyser.zip：包含用于检索APT报告、恶意软件家族、国家及MITRE攻击技术的源代码。若匹配到相关内容，则会更新extracted-iocs.zip中的文件。

10. cti-transformation-app.zip：包含用于将extracted-iocs.zip中的文件转换为网络威胁情报（Cyber Threat Intelligence, CTI）三元组，并将其存储至Neo4j图数据库的源代码。

11. graph-db-backup.zip：包含Neo4j Docker容器的卷文件夹。将该文件夹挂载至Docker容器后，即可通过Neo4j Web界面访问全部CTI数据库。以下为启动挂载该文件夹的Neo4j Docker容器的命令：
docker run -d --publish=7474:7474 --publish=7687:7687 --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/data:/data --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/plugins:/plugins --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/logs:/logs --volume={PATH_TO_VOLUME}/DEVIL_NEO4J_VOLUME/neo4j/conf:/conf --env 'NEO4J_PLUGINS=["apoc","graph-data-science"]' --env NEO4J_apoc_export_file_enabled=true --env NEO4J_apoc_import_file_enabled=true --env NEO4J_apoc_import_file_use__neo4j__config=true --env=NEO4J_AUTH=none neo4j:5.13.0
Web界面地址：http://localhost:7474
用户名：neo4j
密码：neo4j