VPN-nonVPN-Dataset

A flow-level dataset of WireGuard tunnel traffic with matched encrypted-side features and application labels A flow-level dataset derived from paired captures on both sides of a WireGuard VPN tunnel. Each flow combines NFStream-exported pre-tunnel attributes (including application labels from deep packet inspection) with encrypted-side statistics computed from matched WireGuard transport packets. Dataset summary Property Value Capture sessions 2 Total duration ~80 hours Total flows 226,454 (122,975 + 103,479) Columns per flow 123 Unique application names 81 / 75 per session Unique application categories 20 / 19 per session Protocol split ~47% TCP, ~53% UDP Capture location Budapest, Hungary VPN endpoint Surfshark WireGuard, Prague, Czech Republic Active devices 10 Repository structure VPN-nonVPN-Dataset data session1_flows.parquet: Session 1 flow records 122,975 rows session2_flows.parquet : Session 2 flow records 103,479 rows code packet_matching.py : Phase 3 inner outer packet matching flow_matching.py: Phase 3 packet to flow assignment and aggregation Data files Both Parquet files share the same 123-column schema. Each row corresponds to one bidirectional flow. File Rows Capture date Duration session1_flows.parquet 122,975 2025-12-01 ~48 h session2_flows.parquet 103,479 2025-12-04 ~32 h Column schema Each flow record contains 123 columns organized into three groups. NFStream flow attributes (columns 1-89) Standard flow fields exported by NFStream, including: Identifiers: id, expiration_id, src_ip, dst_ip, src_port, dst_port, protocol, ip_version Timing: bidirectional_first_seen_ms, bidirectional_last_seen_ms, bidirectional_duration_ms, and per-direction equivalents (src2dst_*, dst2src_*) Volume: bidirectional_packets, bidirectional_bytes, and per-direction equivalents Packet size statistics: bidirectional_min_ps, bidirectional_mean_ps, bidirectional_stddev_ps, bidirectional_max_ps, and per-direction equivalents Inter-arrival time statistics: bidirectional_min_piat_ms, bidirectional_mean_piat_ms, bidirectional_stddev_piat_ms, bidirectional_max_piat_ms, and per-direction equivalents TCP flags: per-flag packet counts (SYN, ACK, FIN, RST, PSH, URG, CWR, ECE) for bidirectional and per-direction SPLT arrays (first 255 packets per flow): splt_direction — packet direction sequence splt_ps — packet size sequence (bytes) splt_piat_ms — inter-arrival time sequence (milliseconds) Application labels: application_name — nDPI-assigned application name (e.g., DNS, TLS.Facebook, QUIC) application_category_name — nDPI-assigned category (e.g., Web, Network, Chat) application_is_guessed, application_confidence DPI metadata: requested_server_name, client_fingerprint, server_fingerprint, user_agent, content_type Flow assignment metadata (columns 90-94) flow_id — flow identifier flow_start_ms, flow_end_ms — flow time boundaries in milliseconds k5_fwd, k5_rev — forward and reverse 5-tuple keys Encrypted-side derived features (columns 95-123) Statistics computed from matched WireGuard tunnel packets (the table summarizes the encrypted-side features added to each flow and maps them to the closest NFStream attributes, supporting structured comparison and feature selection for learning tasks based solely on tunnel-side statistics) Column name Description Closest NFStream Column matched_packets Number of matched packet rows assigned to the flow bidirectional_packets outer_bytes Sum of outer_padded_length over matched packets assigned to the flow bidirectional_bytes first_matched_time_ms Minimum inner_time among assigned packet rows, converted to milliseconds bidirectional_first_seen_ms last_matched_time_ms Maximum inner_time among assigned packet rows, converted to milliseconds bidirectional_last_seen_ms outer_duration_ms Inner-timeline matched duration, computed as last_matched_time_ms - first_matched_time_ms bidirectional_duration_ms outer_first_matched_time_ms Minimum outer_time among assigned packet rows, converted to milliseconds bidirectional_first_seen_ms outer_last_matched_time_ms Maximum outer_time among assigned packet rows, converted to milliseconds bidirectional_last_seen_ms outer_capture_duration_ms Outer-timeline matched duration, computed as outer_last_matched_time_ms - outer_first_matched_time_ms bidirectional_duration_ms mean_outer_pkt_size Mean of outer_padded_length over matched packets assigned to the flow bidirectional_mean_ps std_outer_pkt_size Standard deviation of outer_padded_length over matched packets assigned to the flow bidirectional_stddev_ps outer_packet_rate matched_packets divided by outer_duration_ms in seconds bidirectional_packets/ (bidirectional_duration_ms / 1000) outer_byte_rate outer_bytes divided by outer_duration_ms in seconds bidirectional_bytes/ (bidirectional_duration_ms / 1000) outer_bytes_in Sum of outer_padded_length for matched packets with direction = INBOUND dst2src_bytes outer_bytes_out Sum of outer_padded_length for matched packets with direction = OUTBOUND src2dst_bytes outer_min_piat_ms_in Minimum positive inter-arrival time between consecutive matched outer packets in the INBOUND direction, in milliseconds dst2src_min_piat_ms outer_mean_piat_ms_in Mean positive inter-arrival time between consecutive matched outer packets in the INBOUND direction, in milliseconds dst2src_mean_piat_ms outer_stddev_piat_ms_in Standard deviation of positive inter-arrival times in the INBOUND direction, in milliseconds dst2src_stddev_piat_ms outer_max_piat_ms_in Maximum positive inter-arrival time in the INBOUND direction, in milliseconds dst2src_max_piat_ms outer_min_piat_ms_out Minimum positive inter-arrival time between consecutive matched outer packets in the OUTBOUND direction, in milliseconds src2dst_min_piat_ms outer_mean_piat_ms_out Mean positive inter-arrival time between consecutive matched outer packets in the OUTBOUND direction, in milliseconds src2dst_mean_piat_ms outer_stddev_piat_ms_out Standard deviation of positive inter-arrival times in the OUTBOUND direction, in milliseconds src2dst_stddev_piat_ms outer_max_piat_ms_out Maximum positive inter-arrival time in the OUTBOUND direction, in milliseconds src2dst_max_piat_ms outer_min_piat_ms Minimum positive inter-arrival time pooled across directions, in milliseconds bidirectional_min_piat_ms outer_mean_piat_ms Mean positive inter-arrival time pooled across directions, in milliseconds bidirectional_mean_piat_ms outer_stddev_piat_ms Standard deviation of pooled positive inter-arrival times, in milliseconds bidirectional_stddev_piat_ms outer_max_piat_ms Maximum pooled positive inter-arrival time, in milliseconds bidirectional_max_piat_ms mean_size_ratio Mean of size_ratio over matched packets, where size_ratio = outer_padded_length / inner_length No NFStream equivalent (outer-side statistical features) std_size_ratio Standard deviation of size_ratio over matched packets assigned to the flow No NFStream equivalent (outer-side statistical features) max_size_ratio Maximum observed size_ratio over matched packets assigned to the flow No NFStream equivalent (outer-side statistical features) Processing code The code directory contains the scripts used to produce the dataset from raw PCAP captures. These are provided for transparency and reproducibility. packet_matching.py Matches inner-side pre-tunnel packets to outer-side encrypted WireGuard transport data packets using time alignment and a padded-length consistency rule. Requirements: Python 3.8+, tshark (Wireshark CLI) python code/packet_matching.py \ --inner inner_capture.pcap \ --outer outer_capture.pcap \ --time-tolerance 15 \ --output packet_matches.csv flow_matching.py Assigns matched packets to NFStream-exported flows using 5-tuple keys with temporal and capacity constraints, then aggregates encrypted-side statistics per flow. Requirements: Python 3.8+, pandas, numpy python code/flow_matching.py \ --packets packet_matches.csv \ --flows nfstream_flows.csv \ --output session_flows Dataset generation pipeline The dataset was produced through a three-phase pipeline: Traffic capture: Paired PCAP captures on the WireGuard tunnel interface (inner, pre-tunnel) and via an inline network TAP (outer, encrypted). NIC offloads were disabled. Nanosecond timestamp precision was enabled. PCAP cleaning: Inner captures were filtered to retain only TCP and UDP packets and to remove non-initial IPv4 fragments. Matching, flow export, and aggregation: packet_matching.py — links inner packets to outer WireGuard transport data packets NFStream — exports flow records from cleaned inner PCAPs with application labeling via nDPI flow_matching.py — assigns matched packets to flows and aggregates encrypted-side features Measurement topology Traffic was captured on a residential broadband connection in Budapest, Hungary. A GL.iNet Flint 2 (GL-MT6000) router served as the WireGuard VPN client gateway. A network TAP was placed inline between the router and the ISP router, mirroring encrypted tunnel traffic to a Linux capture host.