cadog - evaluation data

# Cadog evaluation data This repository contains the evaluation dataset for cadog, CAD web viewer. As of GrabCAD, and STEPTools CAD files, they must be downloaded directly from the GrabCAD and STEPTools websites [1-3,6] ## Directory Structure The `evaluation-files` directory includes all relevant geometry files in various formats: - Meshes: `.obj`, `.mtl`, `.glb`- CAD Parts: `.stp`, `.step`- Assemblies: `.obj`, `.step`, `.json`- Metadata: `.txt` part listings ```data/evaluation-files/├── airboat-chunked-4.1-nano.json├── airboat.json├── airboat.obj├── airboat-parts.txt├── as1-ac-214.stp├── boxy_with_cylindricity.stp├── duck.glb├── duck.mtl├── duck.obj├── f1-db-214.stp├── farmbot-genesis-1.5-chunked-4.1-nano.json├── farmbot-genesis-1.5.mtl├── farmbot-genesis-1.5.obj├── farmbot-genesis-1.5.step├── helicopter-rotor-chunked-4.1-nano.json├── io1-ac-214.stp├── olsk-large-laser-v1-chunked-4.1-nano.json├── olsk-large-laser-v1-manual.json├── olsk-large-laser-v1.obj├── olsk-large-laser-v1.step├── olsk-large-laser-v1.txt├── olsk-small-3d-printer-v1-chunked-4.1-nano.json├── olsk-small-3d-printer-v1.mtl├── olsk-small-3d-printer-v1.obj├── olsk-small-3d-printer-v1.stp├── olsk-small-3d-printer-v1.txt├── teapot.obj├── vex-robot-gripper-chunk-4.1-nano.json└── vex-robot-gripper-parts.txt````Experiement results```data-evaluation-section/├── airboat│ ├── animation-grouping-delta.csv│ ├── llm-eval.csv│ ├── llm_eval_M3.txt│ ├── llm_eval_M5.txt│ ├── llm_eval_raw.json.txt│ ├── llm_label_fixes_M3.json│ ├── llm_label_fixes_M5.json│ ├── paper-ready-summary.csv│ ├── schema-method-3.json│ └── schema-method-5.json├── farmbot│ ├── animation-grouping-delta.csv│ ├── llm-eval.csv│ ├── llm_eval_M3.txt│ ├── llm_eval_M5.txt│ ├── llm_eval_raw.json.txt│ ├── llm_label_fixes_M3.json│ ├── llm_label_fixes_M5.json│ ├── paper-ready-summary.csv│ ├── schema-method-3.json│ └── schema-method-5.json├── helicopter-rotor│ ├── animation-grouping-delta.csv│ ├── llm-eval.csv│ ├── llm_eval_M3.txt│ ├── llm_eval_M5.txt│ ├── llm_eval_raw.json.txt│ ├── llm_label_fixes_M3.json│ ├── llm_label_fixes_M5.json│ ├── paper-ready-summary.csv│ ├── schema-method-3.json│ └── schema-method-5.json├── olsk-large-laser│ ├── animation-grouping-delta.csv│ ├── llm-eval.csv│ ├── llm_eval_M3.txt│ ├── llm_eval_M5.txt│ ├── llm_eval_raw.json.txt│ ├── llm_label_fixes_M3.json│ ├── llm_label_fixes_M5.json│ ├── paper-ready-summary.csv│ ├── schema-method-3.json│ └── schema-method-5.json├── olsk-small-3d-printer│ ├── animation-grouping-delta.csv│ ├── llm-eval.csv│ ├── llm_eval_M3.txt│ ├── llm_eval_M5.txt│ ├── llm_eval_raw.json.txt│ ├── llm_label_fixes_M3.json│ ├── llm_label_fixes_M5.json│ ├── paper-ready-summary.csv│ ├── schema-method-3.json│ └── schema-method-5.json├── parts-airboat.txt├── parts-farmbot.txt├── parts-helicopter-rotor.txt├── parts-olsk-large-laser-v1.txt├── parts-olsk-small-3d-printer.txt├── parts-vex-robot-gripper.txt└── run.org``` ## Dataset Composition The test set is divided into three model types: ### Meshes [7]- Rubber Duck: segmented triangle mesh (`duck.obj`)- Utah Teapot: NURBS surface (`teapot.obj`) ### CAD Parts [6] - `as1-ac-214.stp`- `io1-ac-214.stp`- `f1-db-214.stp`- `boxy_with_cylindricity.stp` ### Assemblies [1-5] [8] - Airboat- Helicopter Rotor- Vex Robot Gripper- OLSK Large Laser V1- OLSK Small 3D Printer V1- Farmbot Genesis 1.5 Includes `.obj`/`.step` models and LLM-generated `.json` schemas. ## Purpose This dataset supports quantitative and qualitative evaluation of hierarchicalanimation schema generation using LLMs. Each assembly varies in size,complexity, and part organization to test robustness and scalability. ## References [1] Khronos Group, “glTF sample models.” [Online]. Available: https://github.com/KhronosGroup/glTF-Sample-Models[2] GrabCAD, “VEX Robot Gripper 3D CAD Model.” Accessed: May 27, 2025. [Online]. Available: https://grabcad.com/library/vex-robot-gripper-1[3] GrabCAD, “Helicopter main rotor 3D CAD Model.” Accessed: May 27, 2025. [Online]. Available: https://grabcad.com/library/helicopter-main-rotor-rotor-principal-d-helicoptere-1[4] D. Ingrassia, A. Porri, L. S. Honda, and W. Schütze, “OLSK-large-laser V1.” Zenodo, Apr. 2023. doi: 10.5281/zenodo.8133916.[5] D. Ingrassia, W. Schütze, A. Porri, and L. S. Honda, “OLSK-small-3D-printer V1.” Zenodo, Apr. 2023. doi: 10.5281/zenodo.8133923.[6] StepTools Inc, “Sample AP214 Data Files.” Accessed: May 27, 2025. [Online]. Available: https://www.steptools.com/docs/stpfiles/ap214/index.html[7] J. Burkardt, “OBJ Files Data samples.” Accessed: May 27, 2025. [Online]. Available: https://people.sc.fsu.edu/~jburkardt/data/obj/obj.html[8] FarmBot Inc., “FarmBot genesis v1.5 CAD models.” 2019. [Online]. Available: https://genesis.farm.bot/v1.5/Extras/cad ## Acknowledgement The project “Fab City - Decentralized Digital Production for Urban Value Creation” is funded by dtec.bw and supported by the European Union - NextGenerationEU.