Zero-shot detection and segmentation of traditional woodcarving tools

Overview This dataset provides the images, manually annotated ground truth, model predictions, segmentation masks, experimental results, source code, and reproducibility materials associated with an exploratory investigation of zero-shot detection and instance segmentation of traditional woodcarving tools. The study evaluates an existing computer vision pipeline combining Grounding DINO for text-conditioned object detection and the Segment Anything Model (SAM) for instance segmentation. Three predefined text-query configurations are examined to investigate how the selected prompts and category-mapping procedures affect the detection and recognition of specialized craft tools. The dataset is intended to support research in computer vision, open-vocabulary object detection, fine-grained tool recognition, explainable evaluation, and the digital documentation of traditional craftsmanship. Rather than establishing the general superiority of a prompting strategy, the experiments provide a reproducible exploratory case study of detection limitations, category-assignment difficulties, and segmentation performance in visually challenging heritage recordings. Dataset Composition The dataset contains 30 selected full-HD frames (1920 × 1080 pixels) extracted from eight source-video assets of the RCI5_Woodcarving_Ego01 collection. Frames were extracted at a nominal rate of one frame per second using FFmpeg and manually selected to maximize variation in visible tools, tool positions and orientations, hand occlusion, and working conditions. The 30 images contain 367 manually annotated tool instances, distributed among three categories: Tool category Annotated instances Chisel 127 Gouge 210 Mallet 30 Total 367 Ground-truth annotations were produced using CVAT and exported in COCO format. Polygon masks describe visible tool regions, including partially occluded and truncated instances. Eftychios Protopapadakis independently reviewed all 367 annotations, with minor polygon-point corrections incorporated into the final ground-truth export. The corrected annotations were used consistently across all three experiments. This quality-control review does not constitute a separate blinded inter-annotator agreement study. Experimental Configurations Three predefined query configurations were evaluated using the Grounding DINO–SAM pipeline: E1 — Craft-contextual descriptions wood carving gouge wooden mallet carving chisel E2 — Simple object names gouge mallet chisel E3 — Visual descriptions curved metal carving blade with wooden handle wooden hammer with cylindrical head straight metal carving blade with wooden handle The archived experimental configurations use Grounding DINO Tiny and SAM ViT-Base model identifiers. The accompanying code and audit documentation describe the recorded prompts, thresholds, matching procedures, normalization rules, and software requirements. The results are derived from these three specific query configurations and should not be interpreted as a general comparison of prompt families. Evaluation Results Performance was evaluated using class-aware and class-agnostic instance matching at a mask intersection-over-union (IoU) threshold of 0.50. The original experimental results are summarized below. Configuration Evaluation TP FP FN F1-score E1 Class-agnostic 35 82 332 14.46% E2 Class-agnostic 38 34 329 17.31% E3 Class-agnostic 32 49 335 14.29% E1 Class-aware 11 106 356 4.55% E2 Class-aware 12 60 355 5.47% E3 Class-aware 8 73 359 3.57% The results demonstrate low recognition coverage across all three configurations. E2 produces fewer false positives under the baseline settings, although its improvement in true positives is small. No correctly matched gouge instances were obtained under the original class-aware evaluation. Additional retrospective analyses document category-normalization behavior, exact raw-label frequencies, object-size effects, qualitative errors, and sensitivity to post-filtering retained predictions. In particular, the relative ranking of the tested configurations changes under stricter score filtering. The archive preserves both the original experimental outputs and the supporting retrospective analyses, allowing researchers to examine these findings and their limitations. Files and Organization The deposit includes the following materials: Core dataset and experiments woodcarving_zero_shot_e1_e2_e3_v5_Full_dataset.zip — Full archived dataset package containing the selected images, ground-truth annotations, experimental outputs, and associated materials. data.zip — Selected woodcarving images and reference annotations. experiments.zip — Experimental records and outputs organized by the E1, E2, and E3 configurations. results.zip — Evaluation results and derived analysis tables. code.zip — Python code and Jupyter notebooks supporting the experimental and evaluation workflows. Documentation and provenance README.md — Entry point describing the dataset, organization, experiment configurations, and usage. ANNOTATION_QC_AND_GUIDELINES.md — Annotation conventions and independent quality-control documentation. FRAME_EXTRACTION_AND_SELECTION.md — Frame extraction, identification, and selection methodology. documentation.zip — Supplementary documentation supporting dataset interpretation and reuse. file_manifest.csv — File inventory for archive inspection. checksums.sha256 — SHA-256 checksums supporting file-integrity verification. Interactive visualization Woodcarving_GroundTruth_Explorer_Zenodo.zip — A standalone interactive HTML viewer containing the 30 selected frames and 367 ground-truth polygon annotations. The viewer allows users to browse images, filter tool categories, inspect individual annotations, adjust overlays, zoom into tool regions, and export selected visualizations. It operates offline without requiring a web server or additional software. Extended reproducibility materials Woodcarving_Reviewer_Audit_Supplement.zip — Supplementary retrospective analysis package containing audit code and supporting outputs for examining mapping behavior, matching results, threshold sensitivity, and other reported diagnostics. The full dataset archive and component archives may contain overlapping materials. The README and file manifests should be consulted to identify the appropriate entry points for reuse. Reproducibility and Limitations The archive supports inspection and retrospective reproduction of the reported evaluation using the preserved predictions, segmentation masks, reference annotations, and evaluation procedures. In October 2026, Eftychios Protopapadakis independently reproduced the evaluation metrics for E1, E2, and E3, obtaining exact agreement with the reported TP, FP, FN, precision, recall, and F1-score values. The historical execution logs from that verification were not preserved. This activity should therefore be distinguished from a fully archived independent reproduction or a new inference run. The study uses a purposively selected exploratory sample rather than an independently validated benchmark. Some visually ambiguous reference instances cannot be confidently classified from the evaluated frame alone. Additionally, the original category-mapping procedures, confidence thresholds, and fixed prompt configurations constrain interpretation of the results. The dataset is provided to encourage further verification, alternative evaluation methodologies, improved recognition pipelines, and additional studies using independently collected and annotated craft recordings. Source Dataset and Attribution The images are derived from: Zabulis, X., Partarakis, N., and Karamaoynas, P. (2025). RCI5_Woodcarving_Ego01. Zenodo. Source DOI: https://doi.org/10.5281/zenodo.14870374 The original source dataset is distributed under the Creative Commons Attribution 4.0 International license (CC BY 4.0). The source collection was accessed approximately in September 2026. Its Zenodo record metadata was retrieved and archived on 20 October 2026 to support provenance verification. The original source-video assets are identified through the archived frame-provenance tables and source-video crosswalk. Licensing The derived dataset, annotations, experimental results, figures, and documentation are distributed under Creative Commons Attribution 4.0 International (CC BY 4.0), subject to the applicable rights of the original material. Original code and notebooks developed for this study are provided under the MIT License. Third-party libraries, pretrained models, and other external components retain their respective licenses. Users must preserve appropriate attribution to the original source dataset and its authors. Funding and Acknowledgments This research was funded by the University of Macedonia, Project No. 82735, through the Research Support Program for Early-Career Faculty Members (1st Cycle), within the project: From Tacit Expertise to Semantic Knowledge: A Framework for Representing Manual Technical Procedures. The authors acknowledge the creators of the original woodcarving recordings and the open-source research communities responsible for Grounding DINO, Segment Anything, and the supporting software. Intended Reuse The dataset may be used for research and education involving: Zero-shot and open-vocabulary object detection. Fine-grained recognition of traditional craft tools. Instance segmentation and reference-mask evaluation. Prompt formulation and category-normalization analysis. Failure analysis, error visualization, and reproducibility studies. Development of AI-assisted workflows for documenting and indexing traditional craft recordings. The dataset is particularly relevant to investigations of the limitations of general-purpose vision models when applied to specialized objects in cultural heritage environments.