Time series dataset for network security situational awareness

In the field of network security situational awareness (NSSA), it is challenging to find a usable dataset. Most of the datasets used in existing research papers are outdated, small, publicly unavailable due to the private infrastructure on which they were created, or unusable for other reasons. This paper presents a new dataset derived from a well-documented and substantial source, suitable for use with neural networks that require larger datasets than classical machine learning approaches. This dataset can help the research community in various ways. The dataset consists of four parts, each containing the time series generated from cybersecurity alerts collected between 2017 and 2018 and between 2023 and 2024. Alerts were collected from the Warden system, which collects and shares information about security events detected by various security systems across multiple organizations. These are data whose labeling is performed by the detection system itself (silver standard). In total, about three billion alerts were collected and processed to time series.

在网络安全态势感知（Network Security Situational Awareness, NSSA）领域，获取可用数据集颇具挑战。现有研究论文中所采用的多数数据集均存在诸多问题：要么过时、规模过小，要么因依托私有基础设施构建而无法公开获取，亦或是因其他缘由无法使用。本文提出一款源自经规范记录且体量充足的数据源的新型数据集，其适配相较于经典机器学习方法需更大数据量的神经网络模型训练。该数据集可从多维度为科研界提供助力。本数据集共分为四个部分，每部分均包含基于2017至2018年以及2023至2024年间采集的网络安全告警生成的时序数据。告警数据采集自Warden系统，该系统可收集并共享由跨多家机构的各类安全系统所检测到的安全事件相关信息。此类数据的标注工作由检测系统自主完成，属于银标准（silver standard）标注数据。本次共采集并处理为时序数据的告警总量约达30亿条。