Annotated Point Clouds and Images from a Spatial-Semantic Perception Pipeline: Robotized Deconstruction at the Reference Construction Site, Aachen

This dataset consists of Semantic Perception Pipeline results in form on annotated pointclouds and images, captured from the robotized deconstruction use case of the TARGET-X project and further processed within the Cluster of Excellence CARE on the CARE Future Technologies Experimental Construction Site at RWTH Aachen Campus Melaten. The site is located at Maria-Lipp-Strasse, 52074 Aachen and operated by Construction Robotics GmbH.The data was captured using a ZED 2i stereo camera at 1920×1080 resolution, 30 fps, with a stereo baseline of 119.76 mm on an Intel Core i7-9850H @ 2.60 GHz, 16 GB DDR4, NVIDIA Quadro T1000, CUDA 12.8, ZED SDK 5.0.2, Ubuntu 24.04 LTS. The perception pipeline is available under a repository at Individualized-Production - spatial-semantic-perception.Each capture is located under output/{date}_{scene}-{time}/, with all files flat in the folder root, named per the EXC CARE filename convention {YYYYMMDD}_{datatype}_{subject}_{version}.{extension}, where {subject} is {scene-tag}-{HHMMSS}-{suffix}: DetectionPTCs_RefSiteAachen/ ├── README.md ← quickstart, what's inside, how to cite ├── LICENSE ← CC-BY-4.0 (data) ├── {date}_{scene}-{time}/ │ ├── {date}_image_{scene}-{time}-raw-left_v1.jpg ← raw left camera image │ ├── {date}_image_{scene}-{time}-raw-right_v1.jpg ← raw right camera image │ ├── {date}_measurement_{scene}-{time}-raw_v1.npy ← raw per-pixel XYZRGBA array (H×W×4 float32) │ ├── {date}_image_{scene}-{time}-labeled_v1.jpg ← annotated image with boxes, labels, and masks │ ├── {date}_pointcloud_{scene}-{time}-segmented_v1.ply ← segmented point cloud with integer label scalar field │ ├── {date}_mask_{scene}-{time}-left-img_v1.npy ← SAM2 segmentation masks on left image (N×H×W bool) │ └── {date}_assessment_{scene}-{time}-meta_v1.json ← detections, prompts, camera pose, camera settings ├── {date}_{scene}-{time}/ │ └── ... └── {date}_{scene}-{time}/ └── ... A single `metadata.json` at the parent directory root holds general context for all captures in JSON format: site location and geolocation, project affiliations (TARGET-X, CARE), hardware/software stack, and full sensor details and calibration. Acknowledgements Grounded SAM 2 — Ren et al., https://github.com/IDEA-Research/Grounded-SAM-2 — Apache 2.0 SAM 2 — Ravi, N., Gabeur, V., Hu, Y.-T., et al. "SAM 2: Segment Anything in Images and Videos." arXiv:2408.00714, 2024. Meta FAIR — Apache 2.0 Grounding DINO — Liu, S., Zeng, Z., Ren, T., et al. "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection." arXiv:2303.05499, 2023. IDEA Research — Apache 2.0 The dataset was recorded during the TARGET-X research project, funded by the Smart Networks and Services Joint Undertaking (SNS JU) under Horizon Europe (Grant Agreement number 101096614). Co-funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the other granting authorities. Neither the European Union nor the granting authority can be held responsible for them. The dataset was annotated and structured as part of the CARE Future Technologies Experimental Construction Site within the Cluster of Excellence CARE — Climate-Neutral and Resource-Efficient Construction, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany's Excellence Strategy – EXC 3115 – 533767731.