flora67/AirScape-Dataset

--- license: mit task_categories: - summarization language: - en tags: - Video - Text - Embodied size_categories: - 10K<n<100K configs: - config_name: benchmark data_files: prompt.parquet --- # [ACM MM'25] AirScape: An Aerial Generative World Model with Motion Controllability This repository contains the dataset introduced in the paper, consisting of two parts: 11k+ motion intention prompts and corresponding video clips. - **Arxiv**: https://arxiv.org/pdf/2507.08885 - **Project**: https://embodiedcity.github.io/AirScape/ - **Code**: https://github.com/EmbodiedCity/AirScape.code ## Dataset Description This dataset is proposed for training and testing of aerial world models. You can use the dataset to train or evaluate the capability of the World Foundation Models to predict future embodied observations based on motion intentions. - **Video Data**: Stored in the `train` and `test` folder. - **Text Data**: Available in the file `prompt.parquet`. The `prompt.parquet` contains the following fields: | Field Name | Description | |:-------------------|:-------------------------------------| | `video_id` | Video name for each motion intention | | `set` | Train set of test set | | `motion_intention` | Motion intention | The proposed dataset exhibits diversity across multiple dimensions, including actions, areas, scenes, and tasks, as illustrated by the examples below. ### Action | | Video | Motion Intention | |:-------------:|:--------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | **Translation** | <img src="assets/dataset_example/action/05638_WebUAV3M_train_minibus_9_1.gif" width="100%"/> | The drone moves rightward while capturing a video of cars moving along the bridge, keeping the bridge centered in its field of view without obvious gimbal adjustment. | | **Rotation** | <img src="assets/dataset_example/action/00001_urbanvideo_train.gif" width="100%"/> | The drone rotates to the right and maintains a steady altitude and camera angle. | | **Compound Movement** | <img src="assets/dataset_example/action/00388_urbanvideo_train.gif" width="100%"/> | The drone, while maintaining its altitude, continuously flies forward towards the parking lot and adjusts its gimbal downward to an overhead view, capturing a top-down view of the parked cars, and eventually stabilizes above the parking lot. | ### Area | | Video | Motion Intention | |:----------------------:|:---------------------------------------------------------------------------------------------------:|:------:| | **Roadside** | <img src="assets/dataset_example/area/00109_urbanvideo_train.gif" width="100%"/> | The drone moved forward steadily while maintaining altitude. Meanwhile, the drone's gimbal has been adjusted downwards to a 45 degree oblique view, capturing a descending viewpoint of a busy urban road surrounded by buildings and vehicles, and ended its final position directly over the road. | | **Tourist Attraction** | <img src="assets/dataset_example/area/05442_WebUAV3M_train_hot-air_balloon_83_0.gif" width="100%"/> | The drone follows the red heart-shaped hot air balloon, gradually rotating leftward, maintaining focus on the balloon without camera gimbal adjustments. | | **Seaside** | <img src="assets/dataset_example/area/05446_WebUAV3M_train_hotel_1_1.gif" width="100%"/> | The drone flies forward while keeping the four buildings in the field of view, no significant altitude change and camera movements. | ### Scene | | Video | Motion Intention | |:------------:|:---------------------------------------------------------------------------------------------------:|:------:| | **Night** | <img src="assets/dataset_example/scene/01909_NAT2021_train_0138tricycle1_3_0.gif" width="100%"/> | The drone flies forward while maintaining its focus on the bridge and the cars below, keeping the camera gimbal stable and centered on the scene. | | **Daytime** | <img src="assets/dataset_example/scene/04170_WebUAV3M_train_container_ship_11_5.gif" width="100%"/> | The drone flies forward while keeping both the Maersk container ship and the tugboat in its field of view, with the camera gimbal slightly adjusting to track them continuously. | | **Snowy** | <img src="assets/dataset_example/scene/09048_WebUAV3M_train_snowmobile_29_1.gif" width="100%"/> | The drone flies upwards and slightly backward while rotating left, keeping a snowmobile in its field of view as it moves around the edge of a forested area on a snow-covered landscape. | ### Task | | Video | Motion Intention | |:--------------:|:---------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | **Navigation** | <img src="assets/dataset_example/task/00925_urbanvideo_test.gif" width="100%"/> | The drone moved forward steadily and turned right slightly, maintaining its altitude, with the camera gimbal slightly tilted downward, capturing a street view with parked cars, trees, buildings, and pedestrians, before coming to a stop over a commercial area. | | **Tracking** | <img src="assets/dataset_example/task/04689_WebUAV3M_train_gaily-painted_pleasure-boat_10_2.gif" width="100%"/> | The drone follows a boat moving forward along the river, maintaining a steady distance while adjusting its position slightly to the left and aligning the camera to keep the boat centered in the field of view. | | **Detection** | <img src="assets/dataset_example/task/08079_WebUAV3M_train_sedan_67_1.gif" width="100%"/> | The drone flies forward while detecting traffic flow on the road. | ## Acknowledgement The drone video data included in this dataset is sourced from three open collections (**All video copyrights belong to their respective dataset owners**): - **UrbanVideo-Bench** - URL: https://huggingface.co/datasets/EmbodiedCity/UrbanVideo-Bench - License: MIT License - **NAT2021** - URL: https://vision4robotics.github.io/NAT2021/ - License: Apache License 2.0 - Notes: Please comply with all terms of the Apache License, Version 2.0 (include a copy of the license and any required NOTICE file, state your modifications, etc.). - **WebUAV-3M** - URL: https://github.com/983632847/WebUAV-3M?tab=readme-ov-file - License: GNU General Public License v3 - Notes: Please comply with all terms of the GNU GPL v3 (distribute derivative works under GPL v3, include the full license text, mark modified files, preserve the warranty disclaimer, etc.). ## Citation ```bibtex @inproceedings{zhao2025airscape, author = {Baining Zhao and Rongze Tang and Mingyuan Jia and Ziyou Wang and Fanhang Man and Xin Zhang and Yu Shang and Weichen Zhang and Wei Wu and Chen Gao and Xinlei Chen and Yong Li}, title = {AirScape: An Aerial Generative World Model with Motion Controllability}, booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia (MM '25)}, year = {2025}, month = {October}, pages = {1--10}, address = {Dublin, Ireland}, publisher = {ACM}, location = {New York, NY, USA}, doi = {10.1145/3746027.3758180}, url = {https://doi.org/10.1145/3746027.3758180} } ```--- 许可证：MIT许可证 任务类别： - 摘要 语言： - 英语 标签： - 视频（Video） - 文本（Text） - 具身（Embodied） 规模类别： - 10K<n<100K 配置项： - 配置名称：基准测试（benchmark） 数据文件：prompt.parquet --- # [ACM MM 2025] AirScape：具备运动可控性的空中生成式世界模型 本仓库包含论文中提出的数据集，由两部分构成：1.1万余条运动意图提示词与对应的视频片段。 - **Arxiv**：https://arxiv.org/pdf/2507.08885 - **项目主页**：https://embodiedcity.github.io/AirScape/ - **代码仓库**：https://github.com/EmbodiedCity/AirScape.code ## 数据集说明 本数据集旨在为空中世界模型的训练与测试提供支撑。您可使用该数据集训练或评估世界基础模型（World Foundation Models）基于运动意图预测未来具身观测结果的能力。 - **视频数据**：存储于`train`与`test`文件夹中。 - **文本数据**：可通过文件`prompt.parquet`获取。该文件包含以下字段： | 字段名 | 描述 | |:-------------------|:-------------------------------------| | `video_id` | 每条运动意图对应的视频名称 | | `set` | 训练集或测试集 | | `motion_intention` | 运动意图 | 本数据集在动作、区域、场景、任务等多个维度展现出丰富多样性，如下方示例所示。 ### 动作 | | 视频示例 | 运动意图描述 | |:-------------:|:--------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | **平移** | <img src="assets/dataset_example/action/05638_WebUAV3M_train_minibus_9_1.gif" width="100%"/> | 无人机向右移动，同时拍摄沿桥梁行驶的车辆画面，始终保持桥梁处于视野中央，云台无明显调整。 | | **旋转** | <img src="assets/dataset_example/action/00001_urbanvideo_train.gif" width="100%"/> | 无人机向右旋转，同时维持稳定的飞行高度与相机角度。 | | **复合运动** | <img src="assets/dataset_example/action/00388_urbanvideo_train.gif" width="100%"/> | 无人机在维持高度的同时，持续向停车场前方飞行，并将云台向下调整至俯视视角，拍摄停放车辆的俯瞰画面，最终稳定停留在停车场上方。 | ### 区域 | | 视频示例 | 运动意图描述 | |:----------------------:|:---------------------------------------------------------------------------------------------------:|:------:| | **路边** | <img src="assets/dataset_example/area/00109_urbanvideo_train.gif" width="100%"/> | 无人机在维持高度的同时稳步向前飞行，同时将云台向下调整至45度斜视角，拍摄逐渐靠近被建筑与车辆环绕的繁忙城市道路的画面，最终停留在道路正上方。 | | **旅游景点** | <img src="assets/dataset_example/area/05442_WebUAV3M_train_hot-air_balloon_83_0.gif" width="100%"/> | 无人机跟随红色心形热气球，逐渐向左旋转，始终将镜头聚焦于热气球，未对相机云台进行调整。 | | **海滨** | <img src="assets/dataset_example/area/05446_WebUAV3M_train_hotel_1_1.gif" width="100%"/> | 无人机向前飞行，同时将四栋建筑保持在视野范围内，飞行高度与相机视角无明显变化。 | ### 场景 | | 视频示例 | 运动意图描述 | |:------------:|:---------------------------------------------------------------------------------------------------:|:------:| | **夜间** | <img src="assets/dataset_example/scene/01909_NAT2021_train_0138tricycle1_3_0.gif" width="100%"/> | 无人机向前飞行，始终将镜头聚焦于桥梁与下方的车辆，保持相机云台稳定，场景居中。 | | **日间** | <img src="assets/dataset_example/scene/04170_WebUAV3M_train_container_ship_11_5.gif" width="100%"/> | 无人机向前飞行，同时将马士基集装箱船与拖轮都保持在视野范围内，相机云台略微调整以持续追踪它们。 | | **积雪场景** | <img src="assets/dataset_example/scene/09048_WebUAV3M_train_snowmobile_29_1.gif" width="100%"/> | 无人机向上并略微向后飞行，同时向左旋转，始终将雪地摩托保持在视野范围内，该雪地摩托正于积雪覆盖的地貌上的森林边缘区域移动。 | ### 任务 | | 视频示例 | 运动意图描述 | |:--------------:|:---------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:| | **导航** | <img src="assets/dataset_example/task/00925_urbanvideo_test.gif" width="100%"/> | 无人机稳步向前飞行并略微右转，维持飞行高度，相机云台略微向下倾斜，拍摄带有停放车辆、树木、建筑与行人的街景，最终停留在商业区上方。 | | **追踪** | <img src="assets/dataset_example/task/04689_WebUAV3M_train_gaily-painted_pleasure-boat_10_2.gif" width="100%"/> | 无人机跟随沿河道向前行驶的船只，保持稳定距离的同时略微向左调整位置，并对齐相机以使船只始终处于视野中央。 | | **检测** | <img src="assets/dataset_example/task/08079_WebUAV3M_train_sedan_67_1.gif" width="100%"/> | 无人机向前飞行，同时检测道路上的交通流量。 | ## 致谢 本数据集包含的无人机视频数据来源于三个开源数据集（所有视频版权归各自数据集所有者所有）： - **UrbanVideo-Bench** - 链接：https://huggingface.co/datasets/EmbodiedCity/UrbanVideo-Bench - 许可证：MIT许可证 - **NAT2021** - 链接：https://vision4robotics.github.io/NAT2021/ - 许可证：Apache许可证2.0 - 说明：请遵守Apache许可证2.0的所有条款（包括附带许可证副本与任何必需的NOTICE文件、说明您的修改内容等）。 - **WebUAV-3M** - 链接：https://github.com/983632847/WebUAV-3M?tab=readme-ov-file - 许可证：GNU通用公共许可证v3 - 说明：请遵守GNU GPL v3的所有条款（根据GPL v3分发衍生作品、附带完整许可证文本、标记修改过的文件、保留免责声明等）。 ## 引用 bibtex @inproceedings{zhao2025airscape, author = {Baining Zhao and Rongze Tang and Mingyuan Jia and Ziyou Wang and Fanhang Man and Xin Zhang and Yu Shang and Weichen Zhang and Wei Wu and Chen Gao and Xinlei Chen and Yong Li}, title = {AirScape: An Aerial Generative World Model with Motion Controllability}, booktitle = {Proceedings of the 33rd ACM International Conference on Multimedia (MM '25)}, year = {2025}, month = {October}, pages = {1--10}, address = {Dublin, Ireland}, publisher = {ACM}, location = {New York, NY, USA}, doi = {10.1145/3746027.3758180}, url = {https://doi.org/10.1145/3746027.3758180} }