llama-farm/drone-planner-dataset

--- license: apache-2.0 tags: - drone - mission-planning - autonomous-systems - instruction-following - json-output language: - en task_categories: - text-generation --- # Drone Planner Dataset — v1 (March 2026) Training and evaluation data for the LlamaFarm drone mission planner. Used to fine-tune Qwen3-4B to 89.3% T1 accuracy on the regression suite. ## Dataset Stats | Split | Examples | Purpose | |-------|----------|---------| | `train.jsonl` | 4,241 | Fine-tuning | | `eval.jsonl` | 431 | Validation during training | | `regression.jsonl` | 196 | Fixed regression suite for scoring | ## Format Each example is a chat-format JSON with system/user/assistant messages: ```json { "messages": [ {"role": "system", "content": "You are a drone mission planner..."}, {"role": "user", "content": "OBSERVATION: Scanning waypoint 8 of 15. Battery 74%."}, {"role": "assistant", "content": "{\"reasoning\": \"...\", \"next_actions\": [...], ...}"} ] } ``` The assistant output is always a valid JSON object with 4 fields: - `reasoning` — 1-2 sentence situation analysis - `next_actions` — list of canonical action strings - `memory_update` — string for context retention - `mission_status` — one of: `Continuing`, `Completed`, `Aborted`, `Paused`, `RTB`, `Emergency` ## Evaluation Methodology **T0:** JSON schema validation — output must be valid JSON with correct types, required fields, and valid enum values. Threshold: 100% required. **T1 (json_semantic):** - `mission_status` — exact string match (enum) - All other fields — semantic equivalence judged by Claude ## Data Quality This dataset underwent extensive manual curation: - All training examples validated against regression ground truth (0 contradictions) - Action vocabulary standardized across all examples - Grounding rules enforced: model only references facts present in the input - Scan/waypoint disambiguation: "Continue scan" only when no explicit waypoint number; "Continue to next waypoint" only when "waypoint X of Y" appears in input - Wind/battery safety thresholds verified against all 196 regression cases - Duplicate examples removed ## Key Decision Rules Encoded | Situation | Expected output | |-----------|----------------| | Battery <20% | `mission_status: Emergency`, `Emergency land` | | Battery 20-30% | `mission_status: RTB`, `Return to home` | | Wind >35mph alone | `mission_status: RTB` | | Wind 33-35mph | `mission_status: RTB` | | Wind ~34mph | `mission_status: Paused`, `Hold position` | | Wind ≤20mph | `mission_status: Continuing` | | Signal 17% + battery 96% | `mission_status: Continuing` (signal alone not RTB) | | Signal 10% + battery 27% | `mission_status: RTB`, `Begin RTB` + `Reduce distance` | | Law enforcement | `mission_status: Aborted` | | FAA/TFR/NOTAM | `mission_status: Aborted` | | Property boundary noted, not crossed | `mission_status: Continuing` | | Property boundary crossed | `mission_status: RTB` | | Input has "waypoint X of Y" | `Continue to next waypoint`, `Scanning wp X/Y` memory | | Generic scan (no waypoint number) | `Continue scan`, `Scanning scan in progress` memory | ## Part of LlamaFarm This dataset trains the mission planner in the [LlamaFarm](https://llama.farm) autonomous drone system.license: Apache-2.0 tags: - 无人机（drone） - 任务规划（mission-planning） - 自主系统（autonomous-systems） - 指令跟随（instruction-following） - JSON输出（json-output） language: - 英语（en） task_categories: - 文本生成（text-generation） # 无人机规划师数据集——v1版（2026年3月） 本数据集为LlamaFarm无人机任务规划器提供训练与评估数据，用于对Qwen3-4B大语言模型（Large Language Model, LLM）进行微调，使其在回归测试套件上的T1准确率达到89.3%。 ## 数据集统计 | 数据集拆分 | 样本数量 | 用途 | |-------|----------|---------| | `train.jsonl` | 4,241 | 模型微调 | | `eval.jsonl` | 431 | 训练期间的验证 | | `regression.jsonl` | 196 | 用于评分的固定回归测试套件 | ## 数据格式 每个样本均为采用对话格式的JSON文件，包含系统（system）、用户（user）、助手（assistant）三类消息： json { "messages": [ {"role": "system", "content": "您是一名无人机任务规划师……"}, {"role": "user", "content": "观测：正在扫描第8/15个航点。剩余电量74%。"}, {"role": "assistant", "content": "{"reasoning": "……", "next_actions": [...], ...}"} ] } 助手输出始终为包含4个字段的合法JSON对象： - `reasoning`：1至2句情境分析内容 - `next_actions`：标准化动作字符串列表 - `memory_update`：用于上下文留存的字符串 - `mission_status`：枚举值之一，可选为：进行中（Continuing）、已完成（Completed）、已中止（Aborted）、已暂停（Paused）、返航（RTB）、紧急情况（Emergency） ## 评估方法 **T0：JSON Schema验证**：输出必须为合法JSON格式，需具备正确的数据类型、必填字段以及合法的枚举值，合格阈值为100%。 **T1（json_semantic）**： - `mission_status`：需与枚举值完全字符串匹配 - 其余所有字段：语义等价性由Claude进行判定 ## 数据质量 本数据集经过了严格的人工审核与整理： - 所有训练样本均与回归测试集的真实标签进行校验，无任何矛盾内容 - 所有样本的动作词汇均已标准化 - 遵循锚定（grounding）规则：模型仅能引用输入中出现的事实信息 - 扫描/航点歧义处理：仅当输入未明确给出航点编号时，使用“继续扫描”；仅当输入包含“waypoint X of Y”格式内容时，使用“前往下一个航点” - 所有196个回归测试样本均已校验风速/电池安全阈值 - 已移除重复样本 ## 内置关键决策规则 | 场景 | 预期输出 | |-----------|----------------| | 剩余电量<20% | `mission_status: 紧急情况（Emergency）`，`紧急降落` | | 剩余电量20%-30% | `mission_status: 返航（RTB）`，`返回起飞点` | | 单独风速>35英里/小时 | `mission_status: 返航（RTB）` | | 风速33-35英里/小时 | `mission_status: 返航（RTB）` | | 风速约34英里/小时 | `mission_status: 暂停（Paused）`，`保持位置` | | 风速≤20英里/小时 | `mission_status: 进行中（Continuing）` | | 信号强度17%+剩余电量96% | `mission_status: 进行中（Continuing）`（仅信号强度不足不会触发返航） | | 信号强度10%+剩余电量27% | `mission_status: 返航（RTB）`，`启动返航流程` + `缩小距离` | | 遭遇执法行动 | `mission_status: 已中止（Aborted）` | | 遭遇美国联邦航空管理局（Federal Aviation Administration, FAA）临时飞行限制（Temporary Flight Restriction, TFR）/航行通告（Notice to Air Missions, NOTAM） | `mission_status: 已中止（Aborted）` | | 检测到空域边界但未越界 | `mission_status: 进行中（Continuing）` | | 已越界 | `mission_status: 返航（RTB）` | | 输入包含“waypoint X of Y”格式内容 | `继续前往下一个航点`，`内存记录：正在扫描wp X/Y` | | 仅通用扫描指令（无航点编号） | `继续扫描`，`内存记录：扫描进行中` | ## 属于LlamaFarm生态 本数据集为LlamaFarm自主无人机系统的一部分，相关系统可通过链接[LlamaFarm](https://llama.farm)访问。