某全国性IPv6核心网络25主节点网络流采样数据

互联网基础行为测量与分析原型系统，能够多源异构支持100G 链路流量采集测量，及PB级测量数据的存储、管理、共享与可视化。因此需要在全国性主干网中部署测量分析平台，开展示范应用，对项目成果进行有效验证。
流测量分系统主要是采集netstream数据、协议解析和分析处理，目的在于帮助网络运行人员实时了解网络流量的特征和分布。该系统支持服务、包长、协议、AS和网段多角度统计分析IPv6流量。
某全国性IPv6核心网络是覆盖全国所有省会城市的高速纯IPv6主干网，在其中30个城市（或机构）建成主干网核心节点，在30个核心节点部署了分布式流测量系统。系统部署示意图如图所示。
2020年到2022年期间，项目组在该全国性网络25个100G主干网节点以及5个10G地区节点的每个分布式流测量系统上，分别执行连续一小时的流数据采集任务，每个节点由一台流量采集服务器（双路Xeon Silver 4210 CPU、64GB内存、12TB硬盘，运行64位CentOS 7版本Linux操作系统）将输入的流量按1:16采样比、Netstream V9格式进行流聚合。
汇集的netstream数据采用NFDUMP格式保存30组（每个核心节点一组，每组12个数据文件，每个文件包含核心节点100G路由器在某天某一小时内每5分钟的Netstream流采样数据），共计360个NFDUMP数据文件，全部数据量约70GB。

This prototype system for Internet basic behavior measurement and analysis supports multi-source heterogeneous 100G link traffic collection and measurement, as well as storage, management, sharing and visualization of PB-level measurement data. Therefore, it is necessary to deploy a measurement and analysis platform on the national backbone network, carry out demonstration applications, and effectively verify the project achievements.
The flow measurement subsystem mainly collects Netstream data, performs protocol parsing and analysis processing, aiming to help network operators understand the characteristics and distribution of network traffic in real time. This system supports multi-angle statistical analysis of IPv6 traffic from the perspectives of services, packet lengths, protocols, AS and network segments.
A national IPv6 core network is a high-speed pure IPv6 backbone network covering all provincial capital cities across the country. Thirty core backbone nodes have been built in 30 cities (or institutions), and distributed flow measurement systems have been deployed on these 30 core nodes. The system deployment schematic diagram is shown in the figure.
From 2020 to 2022, the project team conducted continuous one-hour flow data collection tasks on each distributed flow measurement system of 25 100G backbone nodes and 5 10G regional nodes in this national network. Each node was equipped with a traffic collection server (dual-socket Xeon Silver 4210 CPUs, 64GB RAM, 12TB hard disk, running 64-bit CentOS 7 Linux operating system) to perform flow aggregation on the incoming traffic with a 1:16 sampling ratio and in Netstream V9 format.
The collected Netstream data is saved in 30 groups in NFDUMP format (one group per core node, with 12 data files per group. Each file contains 5-minute Netstream flow sampling data of the 100G router of the core node within a certain hour of a certain day). There are a total of 360 NFDUMP data files, with a total data volume of approximately 70GB.

搜集汇总

背景与挑战

背景概述

该数据集包含2020年至2022年期间，从某全国性纯IPv6核心网络的25个100G主干网节点采集的网络流采样数据，采用Netstream V9格式以1:16采样比进行聚合，并以NFDUMP格式保存，总计约70GB数据。数据集主要用于网络流量测量与分析，支持多角度统计IPv6流量特征，适用于计算机网络研究领域。