deardaniel/secdata-raw

secdata-raw是一个用于微调专注于网络安全语言模型的开源原始语料库，包含从50多个公开来源（如漏洞数据库、威胁情报框架、检测规则语料库、内核安全提交、从业者问答档案和精选的GitHub安全文档）爬取的4,051,139个原始安全文档。数据集以JSONL格式提供，总大小约5.4 GB，许可证为CC BY-SA 4.0。文档涵盖多个主题，包括漏洞披露、威胁情报、检测规则、Linux内核安全、从业者问答和GitHub安全文档等。数据集为多模式结构，不同来源的文档具有不同的字段，但所有文档共享source、text和url字段。数据集目前为原始集合，未进行去重或安全过滤，计划未来版本将转换为指令微调格式。secdata-raw is an open-source raw corpus for fine-tuning cybersecurity-focused language models, containing 4,051,139 raw security documents scraped from 50+ public sources such as vulnerability databases, threat intelligence frameworks, detection rule corpora, kernel security commits, practitioner Q&A archives, and curated security-focused GitHub documentation. The dataset is provided in JSONL format with a total size of ~5.4 GB and is licensed under CC BY-SA 4.0. Documents cover various topics including vulnerability disclosure, threat intelligence, detection rules, Linux kernel security, practitioner Q&A, and GitHub security documentation. The dataset has a multi-schema structure where documents from different sources have different fields, but all share source, text, and url fields. It is currently a raw collection without deduplication or safety filtering, with plans to convert it into instruction-tuning format in future versions.