ActiveHuman Part 2

This is Part 2/2 of the ActiveHuman dataset! Part 1 can be found here. Dataset Description ActiveHuman was generated using Unity's Perception package. It consists of 175428 RGB images and their semantic segmentation counterparts taken at different environments, lighting conditions, camera distances and angles. In total, the dataset contains images for 8 environments, 33 humans, 4 lighting conditions, 7 camera distances (1m-4m) and 36 camera angles (0-360 at 10-degree intervals). The dataset does not include images at every single combination of available camera distances and angles, since for some values the camera would collide with another object or go outside the confines of an environment. As a result, some combinations of camera distances and angles do not exist in the dataset. Alongside each image, 2D Bounding Box, 3D Bounding Box and Keypoint ground truth annotations are also generated via the use of Labelers and are stored as a JSON-based dataset. These Labelers are scripts that are responsible for capturing ground truth annotations for each captured image or frame. Keypoint annotations follow the COCO format defined by the COCO keypoint annotation template offered in the perception package. Folder configuration The dataset consists of 3 folders: JSON Data: Contains all the generated JSON files. RGB Images: Contains the generated RGB images. Semantic Segmentation Images: Contains the generated semantic segmentation images. Essential Terminology Annotation: Recorded data describing a single capture. Capture: One completed rendering process of a Unity sensor which stored the rendered result to data files (e.g. PNG, JPG, etc.). Ego: Object or person on which a collection of sensors is attached to (e.g., if a drone has a camera attached to it, the drone would be the ego and the camera would be the sensor). Ego coordinate system: Coordinates with respect to the ego. Global coordinate system: Coordinates with respect to the global origin in Unity. Sensor: Device that captures the dataset (in this instance the sensor is a camera). Sensor coordinate system: Coordinates with respect to the sensor. Sequence: Time-ordered series of captures. This is very useful for video capture where the time-order relationship of two captures is vital. UIID: Universal Unique Identifier. It is a unique hexadecimal identifier that can represent an individual instance of a capture, ego, sensor, annotation, labeled object or keypoint, or keypoint template. Dataset Data The dataset includes 4 types of JSON annotation files files: annotation_definitions.json: Contains annotation definitions for all of the active Labelers of the simulation stored in an array. Each entry consists of a collection of key-value pairs which describe a particular type of annotation and contain information about that specific annotation describing how its data should be mapped back to labels or objects in the scene. Each entry contains the following key-value pairs: id: Integer identifier of the annotation's definition. name: Annotation name (e.g., keypoints, bounding box, bounding box 3D, semantic segmentation). description: Description of the annotation's specifications. format: Format of the file containing the annotation specifications (e.g., json, PNG). spec: Format-specific specifications for the annotation values generated by each Labeler. Most Labelers generate different annotation specifications in the spec key-value pair: BoundingBox2DLabeler/BoundingBox3DLabeler: label_id: Integer identifier of a label. label_name: String identifier of a label. KeypointLabeler: template_id: Keypoint template UUID. template_name: Name of the keypoint template. key_points: Array containing all the joints defined by the keypoint template. This array includes the key-value pairs: label: Joint label. index: Joint index. color: RGBA values of the keypoint. color_code: Hex color code of the keypoint skeleton: Array containing all the skeleton connections defined by the keypoint template. Each skeleton connection defines a connection between two different joints. This array includes the key-value pairs: label1: Label of the first joint. label2: Label of the second joint. joint1: Index of the first joint. joint2: Index of the second joint. color: RGBA values of the connection. color_code: Hex color code of the connection. SemanticSegmentationLabeler: label_name: String identifier of a label. pixel_value: RGBA values of the label. color_code: Hex color code of the label. captures_xyz.json: Each of these files contains an array of ground truth annotations generated by each active Labeler for each capture separately, as well as extra metadata that describe the state of each active sensor that is present in the scene. Each array entry in the contains the following key-value pairs: id: UUID of the capture. sequence_id: UUID of the sequence. step: Index of the capture within a sequence. timestamp: Timestamp (in ms) since the beginning of a sequence. sensor: Properties of the sensor. This entry contains a collection with the following key-value pairs: sensor_id: Sensor UUID. ego_id: Ego UUID. modality: Modality of the sensor (e.g., camera, radar). translation: 3D vector that describes the sensor's position (in meters) with respect to the global coordinate system. rotation: Quaternion variable that describes the sensor's orientation with respect to the ego coordinate system. camera_intrinsic: matrix containing (if it exists) the camera's intrinsic calibration. projection: Projection type used by the camera (e.g., orthographic, perspective). ego: Attributes of the ego. This entry contains a collection with the following key-value pairs: ego_id: Ego UUID. translation: 3D vector that describes the ego's position (in meters) with respect to the global coordinate system. rotation: Quaternion variable containing the ego's orientation. velocity: 3D vector containing the ego's velocity (in meters per second). acceleration: 3D vector containing the ego's acceleration (in ). format: Format of the file captured by the sensor (e.g., PNG, JPG). annotations: Key-value pair collections, one for each active Labeler. These key-value pairs are as follows: id: Annotation UUID . annotation_definition: Integer identifier of the annotation's definition. filename: Name of the file generated by the Labeler. This entry is only present for Labelers that generate an image. values: List of key-value pairs containing annotation data for the current Labeler. Each Labeler generates different annotation specifications in the values key-value pair: BoundingBox2DLabeler: label_id: Integer identifier of a label. label_name: String identifier of a label. instance_id: UUID of one instance of an object. Each object with the same label that is visible on the same capture has different instance_id values. x: Position of the 2D bounding box on the X axis. y: Position of the 2D bounding box position on the Y axis. width: Width of the 2D bounding box. height: Height of the 2D bounding box. BoundingBox3DLabeler: label_id: Integer identifier of a label. label_name: String identifier of a label. instance_id: UUID of one instance of an object. Each object with the same label that is visible on the same capture has different instance_id values. translation: 3D vector containing the location of the center of the 3D bounding box with respect to the sensor coordinate system (in meters). size: 3D vector containing the size of the 3D bounding box (in meters) rotation: Quaternion variable containing the orientation of the 3D bounding box. velocity: 3D vector containing the velocity of the 3D bounding box (in meters per second). acceleration: 3D vector containing the acceleration of the 3D bounding box acceleration (in ). KeypointLabeler: label_id: Integer identifier of a label. instance_id: UUID of one instance of a joint. Keypoints with the same joint label that are visible on the same capture have different instance_id values. template_id: UUID of the keypoint template. pose: Pose label for that particular capture. keypoints: Array containing the properties of each keypoint. Each keypoint that exists in the keypoint template file is one element of the array. Each entry's contents have as follows: index: Index of the keypoint in the keypoint template file. x: Pixel coordinates of the keypoint on the X axis. y: Pixel coordinates of the keypoint on the Y axis. state: State of the keypoint. The SemanticSegmentationLabeler does not contain a values list. egos.json: Contains collections of key-value pairs for each ego. These include: id: UUID of the ego. description: Description of the ego. sensors.json: Contains collections of key-value pairs for all sensors of the simulation. These include: id: UUID of the sensor. ego_id: UUID of the ego on which the sensor is attached. modality: Modality of the sensor (e.g., camera, radar, sonar). description: Description of the sensor (e.g., camera, radar). Image names The RGB and semantic segmentation images share the same image naming convention. However, the semantic segmentation images also contain the string Semantic_ at the beginning of their filenames. Each RGB image is named "e_h_l_d_r.jpg", where: e denotes the id of the environment. h denotes the id of the person. l denotes the id of the lighting condition. d denotes the camera distance at which the image was captured. r denotes the camera angle at which the image was captured.本数据集为ActiveHuman数据集的第2/2部分！第1部分可在此处获取。 数据集描述 ActiveHuman基于Unity的Perception包生成。该数据集包含175428张RGB图像及其对应的语义分割标注图像，采集自不同环境、光照条件、相机距离与拍摄角度。数据集总计涵盖8种环境、33个人体模型、4种光照条件、7种相机距离（1米至4米）以及36种相机角度（0至360度，间隔10度）。 由于部分参数组合下相机会与场景内其他物体碰撞或超出环境边界，因此并非所有相机距离与角度的组合均被纳入本数据集。 每张图像配套生成2D边界框、3D边界框以及关键点的真值标注，通过Labelers脚本生成，并以基于JSON的数据集格式存储。Labelers是用于为每张采集的图像或帧捕获真值标注的脚本。关键点标注遵循Perception包中提供的COCO关键点标注模板定义的COCO格式。 文件夹配置 本数据集包含3个文件夹： 1. JSON Data：存储所有生成的JSON标注文件 2. RGB Images：存储生成的RGB图像 3. Semantic Segmentation Images：存储生成的语义分割标注图像 核心术语 - 标注（Annotation）：描述单次采集的记录数据。 - 采集（Capture）：Unity传感器完成的单次渲染流程，将渲染结果存储为PNG、JPG等数据文件。 - 主体（Ego）：搭载了一组传感器的物体或人员（例如，若无人机搭载相机，则无人机为主体（Ego），相机为传感器（Sensor））。 - 主体坐标系（Ego coordinate system）：以主体为参照的坐标系。 - 全局坐标系（Global coordinate system）：以Unity中的全局原点为参照的坐标系。 - 传感器（Sensor）：用于采集数据集的设备，本数据集中传感器为相机。 - 传感器坐标系（Sensor coordinate system）：以传感器为参照的坐标系。 - 序列（Sequence）：按时间顺序排列的采集序列，常用于视频采集场景，此时采集的时间顺序关系至关重要。 - UUID（Universally Unique Identifier，通用唯一标识符）：一种唯一的十六进制标识符，可用于标识采集、主体、传感器、标注、标注对象或关键点模板等单个实例。 数据集数据 本数据集包含4类JSON标注文件： 1. annotation_definitions.json：以数组形式存储仿真中所有活跃Labelers的标注定义。每个数组项包含一组键值对，用于描述某一类型的标注，并包含该类标注的相关信息，说明其数据应如何映射回场景中的标签或对象。每个数组项包含以下键值对： - id：标注定义的整数标识符。 - name：标注名称（如关键点、边界框、3D边界框、语义分割）。 - description：标注规格说明。 - format：存储标注规格的文件格式（如json、PNG）。 - spec：各Labeler生成的标注值的格式专属规格。 不同Labeler的spec字段包含不同的标注规格： - BoundingBox2DLabeler/BoundingBox3DLabeler： - label_id：标签的整数标识符。 - label_name：标签的字符串标识符。 - KeypointLabeler： - template_id：关键点模板的UUID。 - template_name：关键点模板的名称。 - key_points：包含关键点模板定义的所有关节的数组。该数组包含以下键值对： - label：关节标签。 - index：关节索引。 - color：关键点的RGBA颜色值。 - color_code：关键点的十六进制颜色码。 - skeleton：包含关键点模板定义的所有骨骼连接的数组。每个骨骼连接定义两个不同关节之间的连接关系，该数组包含以下键值对： - label1：第一个关节的标签。 - label2：第二个关节的标签。 - joint1：第一个关节的索引。 - joint2：第二个关节的索引。 - color：连接的RGBA颜色值。 - color_code：连接的十六进制颜色码。 - SemanticSegmentationLabeler： - label_name：标签的字符串标识符。 - pixel_value：标签的RGBA颜色值。 - color_code：标签的十六进制颜色码。 2. captures_xyz.json：每个此类文件单独存储针对单次采集的各活跃Labelers生成的真值标注数组，以及描述场景中各活跃传感器状态的额外元数据。数组中的每个项包含以下键值对： - id：采集的UUID。 - sequence_id：序列的UUID。 - step：采集在序列中的索引。 - timestamp：序列开始以来的时间戳（单位：毫秒）。 - sensor：传感器属性。该条目包含以下键值对集合： - sensor_id：传感器的UUID。 - ego_id：主体的UUID。 - modality：传感器的模态（如相机、雷达）。 - translation：描述传感器相对于全局坐标系位置的三维向量（单位：米）。 - rotation：描述传感器相对于主体坐标系朝向的四元数。 - camera_intrinsic：（若存在）包含相机内参校准的矩阵。 - projection：相机使用的投影类型（如正交投影、透视投影）。 - ego：主体属性。该条目包含以下键值对集合： - ego_id：主体的UUID。 - translation：描述主体相对于全局坐标系位置的三维向量（单位：米）。 - rotation：描述主体朝向的四元数。 - velocity：描述主体速度的三维向量（单位：米/秒）。 - acceleration：描述主体加速度的三维向量（单位：米/秒²）。 - format：传感器捕获的文件格式（如PNG、JPG）。 - annotations：针对每个活跃Labeler的键值对集合，各键值对如下： - id：标注的UUID。 - annotation_definition：标注定义的整数标识符。 - filename：Labeler生成的文件名称，仅在Labeler生成图像文件时存在该字段。 - values：包含当前Labeler标注数据的键值对列表。 不同Labeler的values字段包含不同的标注规格： - BoundingBox2DLabeler： - label_id：标签的整数标识符。 - label_name：标签的字符串标识符。 - instance_id：对象实例的UUID。同一场景中同一标签的可见对象在同一次采集中具有不同的instance_id。 - x：2D边界框在X轴上的位置。 - y：2D边界框在Y轴上的位置。 - width：2D边界框的宽度。 - height：2D边界框的高度。 - BoundingBox3DLabeler： - label_id：标签的整数标识符。 - label_name：标签的字符串标识符。 - instance_id：对象实例的UUID。同一场景中同一标签的可见对象在同一次采集中具有不同的instance_id。 - translation：描述3D边界框中心相对于传感器坐标系位置的三维向量（单位：米）。 - size：描述3D边界框尺寸的三维向量（单位：米）。 - rotation：描述3D边界框朝向的四元数。 - velocity：描述3D边界框速度的三维向量（单位：米/秒）。 - acceleration：描述3D边界框加速度的三维向量（单位：米/秒²）。 - KeypointLabeler： - label_id：标签的整数标识符。 - instance_id：关节实例的UUID。同一场景中同一关节标签的可见关键点在同一次采集中具有不同的instance_id。 - template_id：关键点模板的UUID。 - pose：本次采集的姿态标签。 - keypoints：包含每个关键点属性的数组。关键点模板文件中定义的每个关键点均为数组的一个元素，每个条目包含以下内容： - index：关键点在关键点模板文件中的索引。 - x：关键点在X轴上的像素坐标。 - y：关键点在Y轴上的像素坐标。 - state：关键点的状态。 注意：SemanticSegmentationLabeler不包含values列表。 3. egos.json：包含每个主体的键值对集合，包括： - id：主体的UUID。 - description：主体的描述。 4. sensors.json：包含仿真中所有传感器的键值对集合，包括： - id：传感器的UUID。 - ego_id：传感器所搭载的主体的UUID。 - modality：传感器的模态（如相机、雷达、声呐）。 - description：传感器的描述（如相机、雷达）。 图像命名规则 RGB图像与语义分割图像采用相同的命名约定，但语义分割图像的文件名前缀会添加字符串"Semantic_"。每张RGB图像的命名格式为"e_h_l_d_r.jpg"，其中： - e：环境的ID。 - h：人物的ID。 - l：光照条件的ID。 - d：拍摄图像时的相机距离。 - r：拍摄图像时的相机角度。