Jarrodbarnes/excytin-sft

该数据集包含专家调查痕迹，用于训练模型在ExCyTIn-Bench上进行多步数据库查询和证据分析。ExCyTIn-Bench是一个用于评估LLM代理在网络安全威胁狩猎上的基准。数据集来源为使用DeepSeek-Reasoner在ExCyTIn-Bench TRAIN问题上生成的数据，格式为TRL SFTTrainer消息格式（用户/助手对），任务是通过SQL查询进行多步网络安全事件调查。数据集分为训练集（304个示例）和测试集（34个示例），覆盖了多个事件ID。每个示例包含消息列表、事件ID和意图类型（GOLDEN或HYPOTHESIS）。GOLDEN意图表示遵循真实解决方案的最优调查路径，HYPOTHESIS意图表示明确的假设形成和测试推理。数据集的质量评分为99.1/100，96.2%的痕迹被评为优秀，平均每个痕迹包含7.3个SQL查询。

This dataset contains expert investigation traces for training models to perform multi-step database querying and evidence analysis on ExCyTIn-Bench, a benchmark for evaluating LLM agents on cyber threat hunting. The dataset is generated using DeepSeek-Reasoner on ExCyTIn-Bench TRAIN questions, formatted as TRL SFTTrainer messages (user/assistant pairs), and the task is multi-step cyber incident investigation via SQL queries. The dataset is divided into a training set (304 examples) and a test set (34 examples), covering multiple incident IDs. Each example includes a list of messages, an incident ID, and an intent type (GOLDEN or HYPOTHESIS). The GOLDEN intent represents optimal investigation paths following ground-truth solutions, while the HYPOTHESIS intent represents explicit hypothesis formation and testing reasoning. The dataset has a quality score of 99.1/100, with 96.2% of traces rated as excellent, and an average of 7.3 SQL queries per trace.