Dataset Using TLS Fingerprints for OS Identification in Encrypted Traffic

The dataset consists of data from three different sources; flow records collected from the university backbone network, log entries from the two university DHCP (Dynamic Host Configuration Protocol) servers and a single RADIUS (Remote Authentication Dial In User Service) accounting server. The data was collected from 2019-07-12 00:00 to 2019-07-16 23:59 with a few hours overhead on both sides of the interval for the log entries to cover long connection sessions overlapping to and from the time frame. We measured the flow data from the university uplink to the Internet. In the dataset, we kept only flows with source IP addresses from university wireless networks (Eduroam). The flow data was then enriched with information from DHCP and RADIUS servers to contain ID of the RADIUS session and operating system od the transmitting device as derived from DHCP logs. The dataset is in the form of CSV file with the following information fields important for OS identification: Basic flow features Date flow start - timestamp of flow start Date flow end - timestamp of flow end Src IPv4 - source IPv4 address sPort - source L4 port Dst IPv4 - destination IPv4 address dPort - destination L4 port Extended TCP/IP parameters SYN size - the size of the initial SYN packet of a TCP connection (in bytes) TCP win - value of TCP Window size parameter TCP SYN TTL - observed TTL value HTTP parameters HTTP Host - hostname from the HTTP request HTTP UA OS - OS identification based on user-agent HTTP UA OS MAJ - OS identification based on user-agent HTTP UA OS MIN - OS identification based on user-agent HTTP UA OS BLD - OS identification based on user-agent TLS parameters TLS SNI - Server Name Indication field TLS SNI length - length of SNI in bytes TLS Client Version - TLS client hello Version field Client Cipher Suites - list of supported cipher suites TLS Extension Types - list of extension IDs TLS Extension Lengths - list of extension lengths TLS Elliptic Curves - list of supported curves (or supported groups in TLS1.3) TLS EC Point Formats - list of EC formats Log based extensions Session ID - ID of the session to match flows from one device Ground Truth OS - OS name derived from log data The observed network traffic contains privacy-sensitive information. Hereby, we declare that the monitored data used for our research were processed in accordance with the EU General Data Protection Regulation 2016/679. The published dataset was anonymized with cryptographic means using Crypto-PAn algorithm to preserve both the scientific value and user privacy. When using this dataset, please cite the original work as follows: <pre><code>@inproceedings{lastovicka2020using, title={Using TLS Fingerprints for OS Identification in Encrypted Traffic}, author={La{\v{s}}tovi{\v{c}}ka, Martin and {\v{S}}pa{\v{c}}ek, Stanislav and Velan, Petr and {\v{C}}eleda, Pavel}, booktitle = {2020 IEEE/IFIP Network Operations and Management Symposium (NOMS 2020)}, doi = {http://dx.doi.org/10.1109/NOMS47738.2020.9110319}, keywords = {OS fingerprinting;passive monitoring;IPFIX;TLS}, isbn = {978-1-7281-4973-8}, pages = {1-6}, publisher = {IEEE Xplore Digital Library}, year = {2020} }</code></pre>

本数据集包含三类不同来源的数据：从大学骨干网络采集的网络流记录、两台大学DHCP（动态主机配置协议，Dynamic Host Configuration Protocol）服务器的日志条目，以及一台RADIUS（远程认证拨号用户服务，Remote Authentication Dial In User Service）计费服务器的日志条目。
数据采集时段为2019年7月12日00:00至2019年7月16日23:59；针对日志条目，采集区间在该时段前后额外增加了数小时，以覆盖跨越该时间框架的长连接会话。
本数据集的流数据采集自大学上行链路至互联网的方向，且仅保留源IP地址属于大学无线网络（Eduroam）的流记录。随后，我们通过DHCP与RADIUS服务器的信息对该流数据进行补充丰富，新增了RADIUS会话ID以及从DHCP日志推导得出的传输设备操作系统信息。
本数据集采用CSV文件格式，包含以下用于操作系统识别的关键信息字段：
### 基础流特征
- 流开始日期：流启动的时间戳
- 流结束日期：流结束的时间戳
- 源IPv4：源IPv4地址
- 源L4端口：源传输层四层端口
- 目的IPv4：目的IPv4地址
- 目的L4端口：目的传输层四层端口
### 扩展TCP/IP参数
- SYN包大小：TCP连接初始SYN包的大小（单位：字节）
- TCP窗口大小：TCP窗口大小参数的取值
- TCP SYN TTL：观测到的TTL值
### HTTP参数
- HTTP Host：HTTP请求中的主机名
- HTTP UA OS：基于用户代理（User-Agent）的操作系统识别结果
- HTTP UA OS MAJ：基于用户代理的操作系统主版本识别结果
- HTTP UA OS MIN：基于用户代理的操作系统次版本识别结果
- HTTP UA OS BLD：基于用户代理的操作系统构建版本识别结果
### TLS参数
- TLS SNI：服务器名称指示（Server Name Indication）字段
- TLS SNI长度：SNI字段的字节长度
- TLS客户端版本：TLS客户端Hello消息中的版本字段
- 客户端密码套件：支持的密码套件列表
- TLS扩展类型：扩展ID列表
- TLS扩展长度：扩展长度列表
- TLS椭圆曲线：支持的曲线列表（TLS 1.3中称为支持组）
- TLS EC点格式：椭圆曲线点格式列表
### 基于日志的扩展字段
- 会话ID：用于匹配同一设备产生的流的会话ID
- 基准操作系统（Ground Truth OS）：从日志数据推导得出的操作系统名称

观测到的网络流量包含隐私敏感信息。特此声明，本研究使用的监控数据已按照《欧盟通用数据保护条例2016/679》（EU General Data Protection Regulation 2016/679）进行处理。已发布的数据集采用Crypto-PAn算法通过加密手段完成匿名化，在保障科学研究价值的同时保护用户隐私。
使用本数据集时，请按以下格式引用原文献：
<pre><code>@inproceedings{lastovicka2020using, title={Using TLS Fingerprints for OS Identification in Encrypted Traffic}, author={La{v{s}}tovi{v{c}}ka, Martin and {v{S}}pa{v{c}}ek, Stanislav and Velan, Petr and {v{C}}eleda, Pavel}, booktitle = {2020 IEEE/IFIP Network Operations and Management Symposium (NOMS 2020)}, doi = {http://dx.doi.org/10.1109/NOMS47738.2020.9110319}, keywords = {OS fingerprinting;passive monitoring;IPFIX;TLS}, isbn = {978-1-7281-4973-8}, pages = {1-6}, publisher = {IEEE Xplore Digital Library}, year = {2020} }</code></pre>