Open Vocabulary Attribute Detection (OVAD) Dataset

Introduction

Current detection datasets usually contain various object annotations. Compared to that, there are few detection dataset contains attribute annotations, which is also important for the task of detection. To address this gap, we propose a novel attribute dataset, OVAD, to support training and testing attribute detection comprehensively. OVAD is built on the nuScenes dataset (license: CC BY-NC-SA 4.0), supplementing it with detailed attribute annotations capturing spatial relationships, motion states, and interactions between objects. It is useful for developing and evaluating systems needing to know complex scene dynamics.

To encourage more follow up works on Open Vocabulary Attribute Detection, we are publicly releasing the dataset split used in our paper ("Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes" by Xinhao Xiang, Kuan-Chuan Peng, Suhas Lohit, Michael J. Jones, Jiawei Zhang, BMVC 2025).

Files in the unzipped folder:

OVAD
|---./README.md: This Markdown file
|---OVAD_full: The OVAD dataset
|---|---OVAD_infos_test.pkl
|---|---OVAD_infos_train.pkl
|---|---OVAD_infos_val.pkl
|---OVAD_mini: The mini OVAD dataset
|---|---OVAD_mini_infos_val.pkl
|---|---OVAD_mini_infos_train.pkl
|---|---example_val_0.json

At a Glance

The size of the unzipped dataset is ~2.5GB.

OVAD is build on the nuScenes dataset (license: CC BY-NC-SA 4.0). Please download the dataset from their original repository.

The .pkl files contains the meta information and the data list. It is organized as follows:

1. Metadata

Type: dict

Content:

version: <class 'str'>

2. Infos

Type: list of dict

Each entry store a sample information, it contains:

Key
Type
Description

lidar_path
<class 'str'>
Path to the LiDAR data.

num_features
<class 'int'>
Number of features in the data.

token
<class 'str'>
Unique identifier for the sample.

sweeps
<class 'list'>
List of previous LiDAR frames.

cams
<class 'dict'>
Camera-related information.

lidar2ego_translation
<class 'list'>
Translation from LiDAR to ego-frame.

lidar2ego_rotation
<class 'list'>
Rotation from LiDAR to ego-frame.

ego2global_translation
<class 'list'>
Translation from ego-frame to global.

ego2global_rotation
<class 'list'>
Rotation from ego-frame to global.

timestamp
<class 'int'>
Timestamp of the sample.

gt_spatial_boxes
np.ndarray (num_spat, 7)
Spatial box information.

gt_spatial_names
np.ndarray (num_spat,)
Spatial relationship names.

gt_boxes
np.ndarray (num_obj, 7)
Ground truth 3D boxes.

gt_names
np.ndarray (num_obj,)
Object category names.

gt_attribute_names
<class 'list'>
Attribute names for each object.

gt_velocity
np.ndarray (num_obj, 2)
Object velocities on x and y axises

num_lidar_pts
np.ndarray (num_obj,)
Number of LiDAR points per object.

num_radar_pts
np.ndarray (num_obj,)
Number of radar points per object.

valid_flag
np.ndarray (num_obj,)
Validity flag for objects.

More information for those keys not related to open vocabulary attribute detection could be found in MMdetection3d (license: Apache 2.0).

Example Representation

{
"metadata": {
"version": "v1.0"
},
"infos": [
{
"lidar_path": "path/to/lidar/file.bin",
"num_features": 5,
"token": "37091c75b9704e0daa829ba56dfa0906",
"sweeps": [...],
"cams": {...},
"lidar2ego_translation": [...],
"lidar2ego_rotation": [...],
"ego2global_translation": [...],
"ego2global_rotation": [...],
"timestamp": 1533201470427893,
"gt_spatial_boxes": [[...], [...], ...],
"gt_spatial_names": ["From the perspective of pedestrian, car is behind pedestrian", "...", "..."],
"gt_boxes": [[...], [...], ...],
"gt_names": ["car", "pedestrian", ...],
"gt_attribute_names": [["cycle.with_rider"], ["pedestrian.standing"], ...],
"gt_velocity": [[0.0, 1.2], [1.1, -0.5], ...],
"num_lidar_pts": [12, 8, ...],
"num_radar_pts": [5, 3, ...],
"valid_flag": [True, False, ...]
},
{...}
]
}

The 'example_val_0.json' file shows the comprehensive example of the one entry data under the "infos" key. It is the first entry data in the validation set.

Citation

If you use the OVAD dataset in your research, please cite our paper:

@inproceedings{yang2025ltoad,
author = {Xiang, Xinhao and Peng, Kuan-Chuan and Lohit, Suhas and Jones, Michael J. and Zhang, Jiawei},
title = {Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes},
booktitle = {The British Machine Vision Conference (BMVC)},
year = {2025}
}

License

The OVAD dataset is released under CC-BY-NC-SA-4.0 license. For the images in the nuScenes dataset, please refer to their website for their copyright and license terms.

Created by Mitsubishi Electric Research Laboratories (MERL), 2024-2025

SPDX-License-Identifier: CC-BY-NC-SA-4.0

# 引言

当前主流的目标检测数据集通常仅包含各类目标的标注信息，而兼具属性标注的检测数据集相对匮乏，尽管属性标注对于检测任务同样至关重要。为填补这一研究空白，我们构建了一款全新的属性检测数据集OVAD，可全面支撑属性检测任务的训练与测试工作。OVAD基于nuScenes数据集（许可协议：CC BY-NC-SA 4.0）构建，并补充了涵盖空间关系、运动状态以及目标间交互的精细化属性标注，可用于开发和评估需要理解复杂场景动态的系统。

为促进开放词汇属性检测（Open Vocabulary Attribute Detection）领域的更多后续研究，我们公开发布了本论文《面向属性的开放词汇多模态3D目标检测（Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes）》（作者：Xinhao Xiang、Kuan-Chuan Peng、Suhas Lohit、Michael J. Jones、Jiawei Zhang，发表于英国机器视觉会议（British Machine Vision Conference, BMVC）2025）中使用的数据集划分方案。

## 解压后文件夹内文件结构
OVAD
|---./README.md：本Markdown说明文档
|---OVAD_full：完整OVAD数据集
|---|---OVAD_infos_test.pkl
|---|---OVAD_infos_train.pkl
|---|---OVAD_infos_val.pkl
|---OVAD_mini：精简版OVAD数据集
|---|---OVAD_mini_infos_val.pkl
|---|---OVAD_mini_infos_train.pkl
|---|---example_val_0.json

## 概览
解压后数据集总大小约2.5GB。
OVAD基于nuScenes数据集构建（许可协议：CC BY-NC-SA 4.0），请从其官方仓库下载原始数据集。
.pkl格式文件包含元信息与数据列表，其组织形式如下：
1. 元数据（Metadata）
类型：字典（dict）
内容：
version：<class 'str'>，即版本字符串

2. 数据条目（Infos）
类型：字典列表（list of dict）
每个条目存储一个样本的完整信息，包含以下字段：

| 键名 | 类型 | 说明 |
|------|------|------|
| lidar_path | <class 'str'> | 激光雷达数据文件路径 |
| num_features | <class 'int'> | 数据的特征维度总数 |
| token | <class 'str'> | 样本唯一标识符 |
| sweeps | <class 'list'> | 历史激光雷达帧列表 |
| cams | <class 'dict'> | 相机相关信息 |
| lidar2ego_translation | <class 'list'> | 激光雷达到自车坐标系的平移变换参数 |
| lidar2ego_rotation | <class 'list'> | 激光雷达到自车坐标系的旋转变换参数 |
| ego2global_translation | <class 'list'> | 自车坐标系到全局坐标系的平移变换参数 |
| ego2global_rotation | <class 'list'> | 自车坐标系到全局坐标系的旋转变换参数 |
| timestamp | <class 'int'> | 样本时间戳 |
| gt_spatial_boxes | np.ndarray (num_spat, 7) | 空间框标注信息 |
| gt_spatial_names | np.ndarray (num_spat,) | 空间关系名称 |
| gt_boxes | np.ndarray (num_obj, 7) | 真实3D目标框标注 |
| gt_names | np.ndarray (num_obj,) | 目标类别名称 |
| gt_attribute_names | <class 'list'> | 每个目标对应的属性名称列表 |
| gt_velocity | np.ndarray (num_obj, 2) | 目标在x、y轴上的运动速度 |
| num_lidar_pts | np.ndarray (num_obj,) | 每个目标关联的激光雷达点数 |
| num_radar_pts | np.ndarray (num_obj,) | 每个目标关联的雷达点数 |
| valid_flag | np.ndarray (num_obj,) | 目标有效性标记 |

对于与开放词汇属性检测无关的字段的更多信息，可参考MMdetection3d（许可协议：Apache 2.0）。

## 示例表示
以下为单条样本数据的完整示例格式：
json
{
"metadata": {
"version": "v1.0"
},
"infos": [
{
"lidar_path": "path/to/lidar/file.bin",
"num_features": 5,
"token": "37091c75b9704e0daa829ba56dfa0906",
"sweeps": [...],
"cams": {...},
"lidar2ego_translation": [...],
"lidar2ego_rotation": [...],
"ego2global_translation": [...],
"ego2global_rotation": [...],
"timestamp": 1533201470427893,
"gt_spatial_boxes": [[...], [...], ...],
"gt_spatial_names": ["From the perspective of pedestrian, car is behind pedestrian", "...", "..."],
"gt_boxes": [[...], [...], ...],
"gt_names": ["car", "pedestrian", ...],
"gt_attribute_names": [["cycle.with_rider"], ["pedestrian.standing"], ...],
"gt_velocity": [[0.0, 1.2], [1.1, -0.5], ...],
"num_lidar_pts": [12, 8, ...],
"num_radar_pts": [5, 3, ...],
"valid_flag": [True, False, ...]
},
{...}
]
}

其中`example_val_0.json`文件展示了`infos`键下单条样本数据的完整示例，对应验证集的第一条样本数据。

## 引用
若您在研究工作中使用OVAD数据集，请引用以下论文：
bibtex
@inproceedings{xiang2025towards,
author = {Xiang, Xinhao and Peng, Kuan-Chuan and Lohit, Suhas and Jones, Michael J. and Zhang, Jiawei},
title = {Towards Open-Vocabulary Multimodal 3D Object Detection with Attributes},
booktitle = {The British Machine Vision Conference (BMVC)},
year = {2025}
}

## 许可协议
OVAD数据集采用CC-BY-NC-SA-4.0许可协议发布。对于nuScenes数据集中的图像，请参考其官方网站的版权与许可条款。
本数据集由三菱电机研究实验室（Mitsubishi Electric Research Laboratories, MERL）于2024-2025年创建
SPDX-License-Identifier: CC-BY-NC-SA-4.0