chandupulluru33/network-traffic-anomaly

--- license: mit --- # Network Traffic Anomaly Dataset A processed and curated dataset for **network traffic anomaly detection**, derived from the **CSE‑CIC‑IDS2018** intrusion detection dataset. This dataset is designed for machine learning and deep learning research on **network security, intrusion detection, and anomaly detection**. Dataset link: https://huggingface.co/datasets/abmallick/network-traffic-anomaly > **Important:** > This dataset is created from the **CSE‑CIC‑IDS2018 dataset** and restructured to be easier to use for modern ML workflows. --- ## 📊 Dataset Overview - **Name:** Network Traffic Anomaly Dataset - **Author:** Abhinav Mallick - **Source Dataset:** CSE‑CIC‑IDS2018 - **Domain:** Network Security / Intrusion Detection - **Modalities:** Tabular - **Format:** Parquet - **License:** MIT --- ## 🧠 Motivation Raw intrusion detection datasets like CSE‑CIC‑IDS2018 are large, fragmented across multiple files, and often difficult to use directly for ML experiments. This dataset was created to: - Simplify access to IDS2018 data - Provide a clean, ML‑ready format - Enable rapid experimentation for anomaly detection models - Support both classical ML and deep learning pipelines It is suitable for **binary anomaly detection** as well as **multi‑class attack classification**. --- ## 📦 Source Dataset: CSE‑CIC‑IDS2018 The original **CSE‑CIC‑IDS2018** dataset was created by the Canadian Institute for Cybersecurity (CIC) and contains realistic benign and malicious network traffic captured over multiple days. Key characteristics of the original dataset: - Realistic enterprise network traffic - Multiple attack categories (DoS, DDoS, brute force, infiltration, botnet, etc.) - Flow‑based statistical features extracted using CICFlowMeter This Hugging Face dataset is a **processed and consolidated version** of that data. --- ## 📋 Features / Columns Each row represents a **network flow** with extracted statistical features. Typical feature categories include: - Flow duration and packet counts - Forward and backward packet statistics - Packet length statistics - Inter‑arrival times - Header and flag features - Byte and packet rate metrics ### Key Columns | Column | Description | |------|------------| | `label` | Target label (benign / attack or anomaly class) | | `attack_type` | Specific attack category (if available) | | `flow_duration` | Duration of the network flow | | `total_fwd_packets` | Total forward packets | | `total_bwd_packets` | Total backward packets | | `flow_bytes_per_sec` | Bytes transferred per second | | `flow_packets_per_sec` | Packets per second | | `packet_length_mean` | Mean packet length | | `packet_length_std` | Packet length standard deviation | | `iat_mean` | Mean inter‑arrival time | | `iat_std` | Inter‑arrival time standard deviation | | `split` | Dataset split (`train` / `val` / `test`) | > Exact columns may vary depending on preprocessing and feature selection. --- ## 🧩 Labels Depending on usage, labels can be interpreted as: ### Binary Classification - **0:** Benign traffic - **1:** Anomalous / Malicious traffic ### Multi‑Class Classification - Benign - DoS / DDoS - Brute Force - Botnet - Infiltration - Web attacks - Other attack types Users are free to remap labels based on their modeling needs. --- ## 🚀 Quick Start ### Installation ```bash pip install datasets pandas pyarrow ``` ### Load Dataset ```python from datasets import load_dataset dataset = load_dataset( "abmallick/network-traffic-anomaly", split="train" ) print(dataset[0]) ``` ### Convert to Pandas ```python df = dataset.to_pandas() df.head() ``` --- ## 📈 Example Use Cases - Network intrusion detection systems (IDS) - Anomaly detection using autoencoders or isolation forests - Supervised attack classification models - Benchmarking ML models on real‑world network traffic - Security analytics and SOC research --- ## 🧪 Suggested Evaluation Metrics - Accuracy - Precision / Recall - F1‑score - ROC‑AUC - False Positive Rate (critical for IDS systems) --- ## 📚 Citation If you use this dataset, please cite both this dataset and the original source: ### This Dataset ```bibtex @misc{mallick2025networktraffic, title={Network Traffic Anomaly Dataset}, author={Mallick, Abhinav}, year={2025}, publisher={Hugging Face}, url={https://huggingface.co/datasets/abmallick/network-traffic-anomaly} } ``` ### Original Dataset (CSE‑CIC‑IDS2018) ```bibtex @dataset{cse_cic_ids2018, title={CSE-CIC-IDS2018: A Large Scale Dataset for Intrusion Detection Systems}, author={Sharafaldin, Iman and Lashkari, Arash Habibi and Ghorbani, Ali A.}, year={2018}, publisher={Canadian Institute for Cybersecurity} } ``` --- ## 📄 License This dataset is released under the **MIT License**. The original CSE‑CIC‑IDS2018 dataset is subject to its own licensing terms. --- ## 🤝 Contributions Feedback, issues, and improvements are welcome via the Hugging Face dataset page.--- 许可证：MIT --- # 网络流量异常检测数据集 本数据集为经过预处理与整理的**网络流量异常检测（network traffic anomaly detection）**专用数据集，源自**CSE‑CIC‑IDS2018**入侵检测数据集。 本数据集专为**网络安全、入侵检测与异常检测**领域的机器学习与深度学习研究打造。 数据集链接：https://huggingface.co/datasets/abmallick/network-traffic-anomaly > **重要说明：** > 本数据集源自**CSE‑CIC‑IDS2018**数据集，并经过重构以适配现代机器学习工作流，更易于使用。 --- ## 📊 数据集概览 - **数据集名称：** 网络流量异常检测数据集 - **作者：** 阿比纳夫·马利克（Abhinav Mallick） - **源数据集：** CSE‑CIC‑IDS2018 - **应用领域：** 网络安全 / 入侵检测 - **数据模态：** 表格型数据 - **数据格式：** Parquet - **许可证：** MIT（MIT License） --- ## 🧠 设计动机 诸如CSE‑CIC‑IDS2018这类原始入侵检测数据集通常体量庞大、文件分散，难以直接用于机器学习实验。 本数据集的研发目标如下： - 简化CSE‑CIC‑IDS2018数据集的获取流程 - 提供整洁且适配机器学习的标准格式 - 支持异常检测模型的快速实验验证 - 兼容经典机器学习与深度学习两种工作流 本数据集既可用于**二元异常检测**，也可支持**多分类攻击识别**任务。 --- ## 📦 源数据集：CSE‑CIC‑IDS2018 原始**CSE‑CIC‑IDS2018**数据集由加拿大网络安全研究所（Canadian Institute for Cybersecurity, CIC）制作，包含多日采集的真实良性与恶意网络流量数据。 原始数据集的核心特征如下： - 真实企业级网络流量 - 涵盖多种攻击类型（拒绝服务攻击DoS、分布式拒绝服务攻击DDoS、暴力破解、渗透入侵、僵尸网络等） - 采用CICFlowMeter工具提取基于流量流的统计特征 本Hugging Face数据集即为该原始数据的**预处理与整合版本**。 --- ## 📋 特征与字段 每一行代表一条**网络流量流**，包含已提取的统计特征。 典型特征类别包括： - 流量流时长与数据包计数 - 正向与反向数据包统计信息 - 数据包长度统计信息 - 数据包到达间隔时间 - 包头与标记位特征 - 字节速率与数据包速率指标 ### 核心字段 | 字段名 | 描述 | |------|------------| | `label` | 目标标签（良性流量 / 攻击或异常类别） | | `attack_type` | 具体攻击类别（如可用） | | `flow_duration` | 流量流时长 | | `total_fwd_packets` | 正向总数据包数 | | `total_bwd_packets` | 反向总数据包数 | | `flow_bytes_per_sec` | 每秒传输字节数 | | `flow_packets_per_sec` | 每秒传输数据包数 | | `packet_length_mean` | 数据包平均长度 | | `packet_length_std` | 数据包长度标准差 | | `iat_mean` | 平均到达间隔时间 | | `iat_std` | 到达间隔时间标准差 | | `split` | 数据集划分（`train` / `val` / `test`，即训练集 / 验证集 / 测试集） | > 实际字段可能因预处理流程与特征选择方案有所差异。 --- ## 🧩 标签体系 根据使用场景不同，标签可按以下两种方式解读： ### 二元分类 - **0：** 良性流量 - **1：** 异常 / 恶意流量 ### 多分类 - 良性流量 - 拒绝服务攻击（DoS） / 分布式拒绝服务攻击（DDoS） - 暴力破解攻击 - 僵尸网络攻击 - 渗透入侵 - Web攻击 - 其他攻击类型 用户可根据建模需求自行调整标签映射规则。 --- ## 🚀 快速上手 ### 安装依赖 bash pip install datasets pandas pyarrow ### 加载数据集 python from datasets import load_dataset dataset = load_dataset( "abmallick/network-traffic-anomaly", split="train" ) print(dataset[0]) ### 转换为Pandas数据框 python df = dataset.to_pandas() df.head() --- ## 📈 典型应用场景 - 网络入侵检测系统（Intrusion Detection System, IDS） - 基于自编码器（autoencoder）或孤立森林（isolation forest）的异常检测任务 - 有监督攻击分类模型 - 在真实网络流量数据集上基准测试机器学习模型 - 安全分析与安全运营中心（Security Operations Center, SOC）研究 --- ## 🧪 推荐评估指标 - 准确率 - 精确率 / 召回率 - F1分数 - 受试者工作特征曲线下面积（ROC-AUC） - 假阳性率（对入侵检测系统至关重要） --- ## 📚 引用规范 若您使用本数据集，请同时引用本数据集与原始源数据集： ### 本数据集 bibtex @misc{mallick2025networktraffic, title={Network Traffic Anomaly Dataset}, author={Mallick, Abhinav}, year={2025}, publisher={Hugging Face}, url={https://huggingface.co/datasets/abmallick/network-traffic-anomaly} } ### 原始数据集（CSE‑CIC‑IDS2018） bibtex @dataset{cse_cic_ids2018, title={CSE-CIC-IDS2018: A Large Scale Dataset for Intrusion Detection Systems}, author={Sharafaldin, Iman and Lashkari, Arash Habibi and Ghorbani, Ali A.}, year={2018}, publisher={Canadian Institute for Cybersecurity} } --- ## 📄 许可证 本数据集采用**MIT许可证（MIT License）**发布。 原始CSE‑CIC‑IDS2018数据集需遵循其自身的许可证条款。 --- ## 🤝 贡献与反馈 欢迎通过Hugging Face数据集页面提交反馈、问题与改进建议。