vyykaaa/dataset_all

--- pretty_name: Web Traffic Normal vs Anomalous language: - en license: mit task_categories: - text-classification tags: - cybersecurity - intrusion-detection - web-security - http - anomaly-detection size_categories: - 10K<n<100K --- # Web Traffic Normal vs Anomalous A labeled HTTP request dataset for web traffic classification. This dataset contains raw HTTP requests collected into two classes: - `anomalous`: malicious or attack-like traffic - `normal`: benign traffic ## Important note about data integrity The request payloads are kept as raw text. No normalization, decoding, truncation, lowercasing, header removal, or field cleaning was applied when creating the labeled files. The only added field is `label`. ## Files Recommended files for Hugging Face Datasets: - `web_traffic_labeled_combined.jsonl` - `filtered_anomalous_traffic_labeled.jsonl` - `normalTrafficTest_clean_labeled.jsonl` Also included: - `web_traffic_labeled_combined.csv` - `filtered_anomalous_traffic_labeled.csv` - `normalTrafficTest_clean_labeled.csv` ## Dataset schema Each example has the following fields: - `text` (`string`): full raw HTTP request, including line breaks - `label` (`string`): one of `anomalous` or `normal` ### Example ```json { "text": "GET http://localhost:8080/tienda1/index.jsp HTTP/1.1\nUser-Agent: Mozilla/5.0 ...", "label": "normal" } ``` ## Class distribution - `anomalous`: 2,014 - `normal`: 36,000 - total: 38,014 ## Recommended usage This dataset is suitable for: - web attack detection - anomaly detection - HTTP request classification - WAF / IDS experimentation - baseline text classification benchmarks for cybersecurity ## Loading with `datasets` ### Load the combined JSONL file ```python from datasets import load_dataset dataset = load_dataset("json", data_files="web_traffic_labeled_combined.jsonl") print(dataset["train"][0]) ``` ### Load train/test splits from local files ```python from datasets import load_dataset dataset = load_dataset( "json", data_files={ "train": "web_traffic_labeled_combined.jsonl", }, ) ``` ## Repository layout suggestion ```text . ├── README.md ├── web_traffic_labeled_combined.jsonl ├── filtered_anomalous_traffic_labeled.jsonl ├── normalTrafficTest_clean_labeled.jsonl ├── web_traffic_labeled_combined.csv ├── filtered_anomalous_traffic_labeled.csv └── normalTrafficTest_clean_labeled.csv ``` ## Limitations - The labels are binary only: `normal` and `anomalous` - The dataset may contain duplicated request patterns - The traffic appears to come from a web application testing context and may not cover all modern attack families - Credentials, hostnames, and request contents should be reviewed before production use ## Citation If you publish work based on this dataset, please cite the dataset repository URL.pretty_name: 正常与异常Web流量 language: - 英语 license: MIT task_categories: - 文本分类 tags: - 网络安全 - 入侵检测 - Web安全 - HTTP - 异常检测 size_categories: - 10000 < 样本量 < 100000 --- # 正常与异常Web流量 本数据集为带标注的HTTP请求数据集，用于Web流量分类任务。 本数据集包含两类标注的原始HTTP请求： - `anomalous`（异常）：恶意或类攻击流量 - `normal`（正常）：良性流量 ## 关于数据完整性的重要说明 请求负载保留为原始文本格式，在生成标注文件时未进行任何归一化、解码、截断、小写转换、移除请求头或字段清洗操作，仅新增了`label`标注字段。 ## 文件列表 适用于Hugging Face Datasets的推荐文件包括： - `web_traffic_labeled_combined.jsonl` - `filtered_anomalous_traffic_labeled.jsonl` - `normalTrafficTest_clean_labeled.jsonl` 此外还包含以下文件： - `web_traffic_labeled_combined.csv` - `filtered_anomalous_traffic_labeled.csv` - `normalTrafficTest_clean_labeled.csv` ## 数据集格式规范 每个数据样本包含以下字段： - `text`（字符串类型）：完整的原始HTTP请求，包含换行符 - `label`（字符串类型）：取值为`anomalous`或`normal` ### 数据示例 json { "text": "GET http://localhost:8080/tienda1/index.jsp HTTP/1.1 User-Agent: Mozilla/5.0 ...", "label": "normal" } ## 类别分布 - 异常流量：2014条 - 正常流量：36000条 - 总样本量：38014条 ## 推荐应用场景 本数据集适用于： - Web攻击检测 - 异常流量检测 - HTTP请求分类 - Web应用防火墙（WAF, Web Application Firewall）/入侵检测系统（IDS, Intrusion Detection System）实验 - 网络安全领域文本分类基准测试 ## 使用`datasets`库加载数据集 ### 加载合并后的JSONL文件 python from datasets import load_dataset dataset = load_dataset("json", data_files="web_traffic_labeled_combined.jsonl") print(dataset["train"][0]) ### 从本地文件加载训练划分 python from datasets import load_dataset dataset = load_dataset( "json", data_files={ "train": "web_traffic_labeled_combined.jsonl", }, ) ## 仓库布局建议 text . ├── README.md ├── web_traffic_labeled_combined.jsonl ├── filtered_anomalous_traffic_labeled.jsonl ├── normalTrafficTest_clean_labeled.jsonl ├── web_traffic_labeled_combined.csv ├── filtered_anomalous_traffic_labeled.csv └── normalTrafficTest_clean_labeled.csv ## 局限性说明 - 仅支持二分类标注：`normal`（正常）与`anomalous`（异常） - 数据集中可能存在重复的请求模式 - 采集的流量来自Web应用测试场景，未覆盖所有现代攻击家族 - 在生产环境使用前，需对其中的凭据、主机名及请求内容进行审查 ## 引用说明 若基于本数据集发表研究成果，请引用本数据集仓库的URL。