Data for "Image-based Backbone Reconstruction for Non-Slender Soft Robots"

This dataset provides the data for the forthcoming paper "Image-based Backbone Reconstruction for Non-Slender Soft Robots". The backbone reconstruction method used is based on the method described in Hoffmann et al. [1]. The modifications to this method to support the non-slender soft robot in this dataset are described in the forthcoming paper mentioned above. This dataset holds raw images of pressurized and elongated soft robots and the corresponding reconstructed backbones.

Dataset

The dataset is split into two subsets with similar structure. The first subset is contained in `dataset_01`. The second dataset is contained in `dataset_02`.

Each subset consists of five folders and one schedule file. The schedule file `schedule.csv` contains the index of the schedule entry, the angle α in degree, the pressure of each chamber p_1 to p_3 in bar and if the pressurization is active. Furthermore, the five folders of the subset can be described as follows

- `raw`: Contains the raw cropped images. The filenames are formatted as `CROPPED_C{CAMERA_INDEX}_E{SCHEDULE_ENTRY}.png` with the camera index `CAMERA_INDEX` and the schedule entry `SCHEDULE_ENTRY`.

-`constant_curvature_slender`, `constant_curvature_volumetric`, `cubic_curvature_slender` and `cubic_curvature_volumetric`. These folders contain the actual reconstructed backbones based on the raw data from the `raw` folder. A different reconstruction approach was used in each of these folders  - `constant_curvature_slender` - A constant curvature backbone kinematic based on the slender model, - `constant_curvature_volumetric` - A constant curvature backbone kinematic based on the volumetric model, - `cubic_curvature_slender` - A cubic curvature backbone kinematic based on the slender model, - `cubic_curvature_volumetric` -  A cubic curvature backbone kinematic based on the volumetric model.Each of these folders contain a `data` and `figures` folder. The data folder consists of `PARAMETER_E{SCHEDULE_ENTRY}.json` files listing the optimization parameters for each schedule entry `SCHEDULE_ENTRY` in the JSON format. The `figures` folder contains annotated images of the reconstructed backbone on the cropped raw images. The filenames are structured `ANNOTATED_E{SCHEDULE_ENTRY}_C{CAMERA_INDEX}_EPOCH{EPOCH}.png` with the schedule entry `SCHEDULE_ENTRY`, the camera index `CAMERA_INDEX` and the epoch `EPOCH` of the optimization algorithm.

The optimization parameters include the base position `base_position` of the reconstructed backbone in world coordinates, the coefficients for the curvature polynomials `ux` and `uy`, and the constant coefficient for the elongation polynomial `la`.

Calibration Data

The calibration data is located in the `calibration` folder and consists of multiple `.npy` files in the numpy format. The corresponding camera index for the calibrated camera is abbreviated with `CAMERA_INDEX` in the following:

`C{CAMERA_INDEX}.npy` - Stores the reprojection error, camera matrix, distortion coefficients, rotation, and translation vectors as returned by the `cv2.calibrateCamera` [2] method. 

`C{CAMERA_INDEX}_camera_matrix.npy` - Stores the camera_matrix as returned by the `cv2.calibrateCamera` [2] method. 

`C{CAMERA_INDEX}_distortion_coefficients.npy` - Stores the distortion coefficients as returned by the `cv2.calibrateCamera` [2] method. 

 `C{CAMERA_INDEX}_projection_matrix.npy` - Stores the projection matrix from world space to pixel space based on the stereo camera calibration.

 `STEREO.npy` - Stores the reprojection error, R, T, E, F as returned by the `cv2.stereoCalibrate` [2] method as an object datatype.

Acknowledgement

Funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – 501861263 – SPP2353

References

[1] M. K. Hoffmann, J. Mühlenhoff, Z. Ding, T. Sattel and K. Flaßkamp. An iterative closest point algorithm for marker-free 3D shape registration of continuum robots. arXiv.https://arxiv.org/abs/2405.15336

[2] OpenCV. Camera Calibration and 3D Reconstruction. OpenCV Documentation. https://docs.opencv.org/4.x/d9/d0c/group__calib3d.html, accessed May 27, 2024.