Manipulator Design and Target Localization for Safe Aerial Physical Inspection Mission

Dataset associated with the paper "Manipulator Design and Target Localization for Safe Aerial Physical Inspection Mission"

Content

Content of the repository

training_dataset.zip folder containing the images used for the training of the network

tunnel_rosbag.zip folder containing  rosbag from flight into a tunnel

indoor_rosbag.zip folder containing rosbag from indoor flight

outdoor_rosbag.zip folder containing  rosbag from flight into a industrial enviroment

training_dataset

Each image used for training the network is accompanied by a corresponding JSON file that stores annotation details, including the target's label and position.

File Structure:

Image File .pngContains the visual data used for training.

Annotation File: .jsonStores metadata and parameters, such as the label and target position, associated with the image.

JSON structure

Field
Description
Example

version
Version of the annotation format
3.16.7

flags
Additional metadata or flags (optional)

shapes
List of objects annotated in the image
-

label
Label of the shape
"trg"

line_color
Line color of the shape in RGB format (optional)
[0, 255, 128]

fill_color
Fill color of the shape in RGB format (optional)
[255, 0, 128]

points
Coordinates defining the shape (i.e. rectangle) in pixel format [width, height]
[[x1,y1],[x2,y2]]

shape_type
Type of the shape
"rectangle"

flags
Flags for specific shape (optional)

lineColor
Line color for the shapes (RGB)
[0, 255, 128]

fillColor
Fill color for the shapes (RGB)
[255, 0 ,255]

imagePath
Image file name
"image_0.png"

imageData
Image data

imageHeight
Image height (pixel)
480

imageWidth
Image width (pixel)
640

tunnel_rosbag

Rosbag data recorded during manual flight inside a tunnel, including interactions with three designated targets.

Topic Name
Topic Type
Description

/d400/color/image_raw
sensor_msgs/image
 Color image data from the detection camera.

/d400/depth/image_rect_raw
sensor_msgs/image
Depth image data from the detection camera

/mavros/local_position/odom
nav_msgs/Odometry
Drone position data 

/scan
sensor_msgs/LaserScan
Scans from the 2D LiDAR

/teraranger_evo
sensor_msgs/Range
Altitude measurements from the 1D LiDAR.

indoor_rosbag

Rosbag data recorded during indoor flight in control environment. Data contains comparison between estimated target pose and real target pose (measured using VICON)

Topic Name
Topic Type
Description

/d400/color/image_raw
sensor_msgs/image
 Color image data from the detection camera.

/d400/aligned_depth_to_color/image_raw
sensor_msgs/image
Depth image data from the detection camera

/d400/aligned_depth_to_color/camera_info
sensor_msgs/CameraInfo
Camera parameters from the detection camera

/d400/color/camera_info
sensor_msgs/CameraInfo
Camera parameters from the detection camera

/pixy/vicon/aerotrain/aerotrain/odom
nav_msgs/Odometry
Drone pose (VICON)

/target/debug/percentage
std_msgs/Float64
Percentage of positive target detection over total detection

/target/debug/target_pose_truth
geometry_msgs/PoseStamped
3D measured position of target w.r.t the drone (VICON)

/target/detection
std_msgs/Bool
Positive detection of the target

/target/image_with_detection
sensor_msgs/Image
Color image with detection box

/target/pose
geometry_msgs/Pose
3D estimated position of target w.r.t the drone

outdoor_rosbag

Rosbag data recorded during outdoor flights in an industrial environment.

Topic Name
Topic Type
Description

/d400/color/image_raw
sensor_msgs/image
 Color image data from the detection camera.

/d400/aligned_depth_to_color/image_raw
sensor_msgs/image
Depth image data from the detection camera

/d400/aligned_depth_to_color/camera_info
sensor_msgs/CameraInfo
Camera parameters from the detection camera

/mavros/local_position/odom
nav_msgs/Odometry
Drone position data 

/desired_joint_states
sensor_msgs/JointState
References of robot arm servos

/joint_states
sensor_msgs/JointState
Actual values of robot arm servos

/mavros/state
mavros_msgs/State
PX4 state

/target/depth
std_msgs/Float32
Distance drone to target

/target/detection
std_msgs/Bool
Positive detection of the target

/target/pose
geometry_msgs/Pose
3D position of target w.r.t the drone