helimistry/drone_detections

# UAV Drone Detection and Tracking with YOLO and Kalman Filter

## Overview

This project detects UAV drones in video using a deep learning object detector and tracks them across frames using a Kalman filter. The system processes every `.mp4` file in a given input directory, saves frames containing detections, and produces one tracked output video per input video with bounding boxes and 2D trajectories overlaid.

---

## Test Videos

The following YouTube videos were used as the primary test inputs:

- Video 1: https://www.youtube.com/watch?v=DhmZ6W1UAv4
- Video 2: https://www.youtube.com/watch?v=YrydHPwRelI

### Output Tracking Videos

Replace these with your uploaded YouTube links:

- Output Video 1: https://www.youtube.com/watch?v=hkxeuLE6Gns
- Output Video 2: https://www.youtube.com/watch?v=gO7WVP6hqQo

---

## Dataset Choice

For drone detection, I used the Hugging Face dataset:

- **Dataset:** `ChinnaSAMY1/drone-detection-dataset`
- **Source:** Hugging Face
- **Link** https://huggingface.co/datasets/ChinnaSAMY1/drone-detection-dataset

This dataset was selected because it contains **bounding box annotations for drones themselves**, which matches the assignment requirement. Many aerial vision datasets instead focus on detecting objects *from* drones, which is not the same problem.

The dataset was converted into **YOLO format** for training. The original annotations were provided as bounding boxes, and each bounding box was transformed into normalized YOLO labels:

- class id
- center x
- center y
- width
- height

Since this project only tracks drones, the dataset was treated as a **single-class detection task**:

- `class 0 = drone`

---

## Detector Configuration

The detector used in this project is:

- **Model:** Ultralytics YOLOv8
- **Starting checkpoint:** `yolov8n.pt`

YOLOv8 was chosen because it is easy to train, works well for custom object detection tasks, and integrates cleanly with Python for inference on video frames.

### Training Setup

The detector was fine-tuned on the drone dataset after converting the dataset into YOLO folder structure.

Example configuration:

- image size: `640`
- batch size: `16`
- epochs: `30` for a full run
- smaller debug runs were also used, such as:
- epochs: `3`
- image size: `416`
- batch size: `8`

The final trained model weights were saved as `best.pt` and then used during the video-processing stage.

### Inference Pipeline

For each `.mp4` file in the input directory:

1. Open the video with OpenCV
2. Read frames one by one
3. Run the YOLO detector on each frame
4. Keep detections above the confidence threshold
5. Save any frame containing at least one detection into `detections/`
6. Pass the detections to the Kalman filter tracker
7. Draw:
- bounding box
- track ID
- estimated center point
- 2D trajectory polyline
8. Write the result to an output video

---

## Kalman Filter Tracking

The second part of the project uses a Kalman filter to track the drone across frames.

### State Design

The Kalman filter uses a **constant velocity motion model** with the following state vector:

```text
[x, y, vx, vy]
```

### Noise Parameters

The Kalman filter requires covariance settings for uncertainty.

Initial covariance P

I used a large initial covariance: kf.P *= 500.0

This reflects high uncertainty at the beginning of the track, especially because the initial velocity is unknown.

Measurement noise R

I used:
kf.R = np.array([
[25., 0.],
[0., 25.]
])

This represents moderate uncertainty in detector measurements. Bounding box centers can jitter slightly from frame to frame, so the tracker should not trust every detection perfectly.

Process noise Q

I used:

q = 1.0
kf.Q = np.array([
[q, 0, 0, 0],
[0, q, 0, 0],
[0, 0, q, 0],
[0, 0, 0, q]
])

This allows the tracker to adapt to small motion changes while still favoring smooth trajectories.

### Handling Missing Detections
If no detection is assigned to a track in a frame, the Kalman filter still performs the predict step.

The track remains alive temporarily the predicted state is used for continuity. Each track keeps a missed counter.

If the tracker misses the drone for too many frames in a row, the track is removed.

# 基于YOLO（You Only Look Once）与卡尔曼滤波的无人机检测与跟踪

## 概述
本项目基于深度学习目标检测器实现视频中的无人机（UAV，Unmanned Aerial Vehicle）检测，并通过卡尔曼滤波实现跨帧跟踪。系统可处理指定输入目录下的所有`.mp4`文件，保存包含检测结果的帧，并为每个输入视频生成带边界框与二维轨迹叠加效果的跟踪输出视频。

---

## 测试视频
以下YouTube视频用作主要测试输入：
- 测试视频1：https://www.youtube.com/watch?v=DhmZ6W1UAv4
- 测试视频2：https://www.youtube.com/watch?v=YrydHPwRelI

### 跟踪输出视频
请将此处替换为您上传的YouTube视频链接：
- 输出视频1：https://www.youtube.com/watch?v=hkxeuLE6Gns
- 输出视频2：https://www.youtube.com/watch?v=gO7WVP6hqQo

---

## 数据集选择
本项目选用Hugging Face平台下的如下数据集开展无人机检测任务：
- **数据集名称**：`ChinnaSAMY1/drone-detection-dataset`
- **来源平台**：Hugging Face
- **数据集链接**：https://huggingface.co/datasets/ChinnaSAMY1/drone-detection-dataset

选择该数据集的原因在于其包含**无人机自身的边界框标注**，完全符合本次任务的需求。多数航空视觉数据集均以「从无人机平台视角检测地面/空中目标」为研究方向，与本次任务场景并不一致。

为适配模型训练，该数据集被转换为**YOLO格式**。原始标注以边界框形式提供，经转换后得到标准化的YOLO标签，格式如下：
- 类别ID
- 中心点横坐标
- 中心点纵坐标
- 框宽度
- 框高度

由于本项目仅需跟踪无人机，因此将该数据集视为**单类别目标检测任务**：
- `类别0 = 无人机`

---

## 检测器配置
本项目使用的检测器如下：
- **模型**：Ultralytics YOLOv8
- **初始预训练权重**：`yolov8n.pt`

选用YOLOv8的原因在于其训练流程简便、适配自定义目标检测任务效果优异，且可与Python生态无缝集成，便于开展视频帧推理。

### 训练配置
在将数据集转换为YOLO文件夹格式后，我们在该无人机数据集上对模型进行了微调。
典型训练配置如下：
- 输入图像尺寸：`640`
- 批次大小：`16`
- 训练轮次：完整训练时设置为`30`轮
此外还开展了小型调试训练，例如：
- 训练轮次：`3`轮
- 输入图像尺寸：`416`
- 批次大小：`8`
最终训练得到的最优模型权重保存为`best.pt`，并在视频处理阶段投入使用。

### 推理流程
针对输入目录下的每个`.mp4`文件，执行以下流程：
1. 通过OpenCV加载视频
2. 逐帧读取视频内容
3. 在每一帧上运行YOLO检测器
4. 保留置信度阈值以上的检测结果
5. 将包含至少一个检测目标的帧保存至`detections/`目录
6. 将检测结果传入卡尔曼滤波跟踪器
7. 可视化叠加内容：
- 目标边界框
- 跟踪ID
- 目标估计中心点
- 二维轨迹折线
8. 将处理后的结果写入输出视频文件

---

## 卡尔曼滤波跟踪
本项目的第二部分通过卡尔曼滤波实现无人机的跨帧跟踪。

### 状态设计
卡尔曼滤波采用**恒速度运动模型**，其状态向量定义如下：
text
[x, y, vx, vy]

### 噪声参数设置
卡尔曼滤波需要针对不确定性设置协方差参数。
1. **初始协方差矩阵P**：
本项目设置较大的初始协方差：`kf.P *= 500.0`，以反映跟踪初始阶段的高不确定性，尤其是初始速度未知的场景。
2. **测量噪声矩阵R**：
本项目采用如下配置：
python
kf.R = np.array([
[25., 0.],
[0., 25.]
])

该参数代表检测器测量结果的中等不确定性，由于相邻帧的边界框中心点可能存在小幅抖动，因此跟踪器无需完全信任每一次检测结果。
3. **过程噪声矩阵Q**：
本项目采用如下配置：
python
q = 1.0
kf.Q = np.array([
[q, 0, 0, 0],
[0, q, 0, 0],
[0, 0, q, 0],
[0, 0, 0, q]
])

该设置可使跟踪器适配小幅运动变化，同时仍能保证轨迹的平滑性。

### 丢失检测的处理逻辑
若某一帧中没有检测结果与已有跟踪目标匹配，卡尔曼滤波仍会执行预测步骤。此时将利用预测状态维持跟踪的连续性，每个跟踪目标会维护一个丢失计数。若跟踪器连续多帧未能匹配到目标，则将该跟踪任务移除。