CAIDA UCSD Network Telescope Aggregated Flow Dataset (2008-10-01)

Raw data captured by the UCSD Network Telescope are stored in huge pcap files. In order to enable more efficient data storage, processing, and analysis, these hourly pcap files are post-processed using Corsaro software to extract the most important packet header fields and aggregate data into FlowTuple files. The FlowTuple format includes eight fields: source IP address; destination IP address; source port; destination port; protocol; TCP Flags; TTL and IP length. In the hourly FlowTuple output files, the data are broken into 60-second intervals. Within a given interval, each unique key (a unique combination of the FlowTuple fields) observed in the raw pcap data is reported on a separate line in the FlowTuple format followed by the number of packets in this interval whose header fields match this FlowTuple key. Flows are further subdivided into three FlowTuple classes: backscatter, ICMP Request and "other", and the total number of flows in each class is recorded. These data provide opportunities to study network security-related events (DDoS attacks, worms, etc.) For more information see http://www.caida.org/data/passive/telescope-flowtuple.xml

UCSD网络望远镜捕获的原始数据存储于大型pcap文件中。为实现更高效的数据存储、处理与分析，研究人员使用Corsaro软件对这些按小时生成的pcap文件进行后处理，提取关键报文首部字段，并将数据聚合为流元组（FlowTuple）格式文件。流元组格式包含8个字段：源IP地址、目的IP地址、源端口、目的端口、协议、TCP标志位、TTL（Time To Live，生存时间）以及IP总长度。在按小时生成的流元组输出文件中，数据被划分为60秒的时间间隔。在指定时间间隔内，原始pcap数据中观测到的每个唯一键（即流元组各字段的唯一组合）将以流元组格式单独占一行，其后跟随该间隔内首部字段匹配此流元组键的报文数量。流量还会被进一步划分为三类流元组类别：反向散射流量（backscatter）、ICMP请求流量（ICMP Request）以及“其他”流量，并记录每一类别的总流量数。此类数据为研究网络安全相关事件（如DDoS攻击、蠕虫等）提供了契机。更多详细信息可参阅：http://www.caida.org/data/passive/telescope-flowtuple.xml

搜集汇总

背景与挑战

背景概述

该数据集是CAIDA UCSD网络望远镜聚合流数据集，起始于2008年10月1日，数据收集持续进行。它包含由网络望远镜捕获的未请求流量，以FlowTuple格式按小时聚合，用于研究网络安全事件如DDoS攻击和蠕虫，数据大小约1.5GB且不断增长，属于受限访问的黑洞地址空间数据类别。